We get the following events in the eventviewer. The database is dismounted and after a few seconds the database will mounted and dismounted. We use Exchange server 2016 with CU12. When we check the database with eseutil, the database is in a dirty state. We fix the status to clean shutdown and mount the database, the database will dismount and is in a dirty state.
Event 1: 1002
Unhandled exception (System.InvalidOperationException: Invalid table operation. operationType = Delete, table = PseudoIndexControl, partitionValues = null
at Microsoft.Exchange.Server.Storage.StoreCommonServices.Context.OnBeforeTableAccess(OperationType operationType, Table table, IList`1 partitionValues)
at Microsoft.Exchange.Server.Storage.LogicalDataModel.LogicalContext.OnBeforeTableAccess(OperationType operationType, Table table, IList`1 partitionValues)
at Microsoft.Exchange.Server.Storage.PhysicalAccessJet.JetConnection.GetOpenTable(JET_DBID jetDatabase, Table table, String tableName, IList`1 partitionValues, OperationType operationType, Boolean allowDDL)
at Microsoft.Exchange.Server.Storage.PhysicalAccessJet.JetTableOperator.TryOpenJetCursorIfNecessary(OperationType operationType, Boolean checkForCorruptedPrimaryIndex, Boolean& primaryIndexCorrupted)
at Microsoft.Exchange.Server.Storage.PhysicalAccessJet.JetTableOperator.MoveFirst(Boolean positionForUpdate, OperationType operationType, Int32& rowsSkipped)
at Microsoft.Exchange.Server.Storage.PhysicalAccessJet.JetDeleteOperator.ExecuteScalar()
at Microsoft.Exchange.Server.Storage.LazyIndexing.LogicalIndexCache.LogicalIndexInfo.DeleteIndexDefinition(Context context, MailboxState mailboxState)
at Microsoft.Exchange.Server.Storage.LazyIndexing.LogicalIndexCache.FolderIndexCache.LoadFolderCache(Context context, MailboxState mailboxState)
at Microsoft.Exchange.Server.Storage.LazyIndexing.LogicalIndexCache.FolderIndexCache..ctor(Context context, LogicalIndexCache logicalIndexCache, MailboxState mailboxState, ExchangeId folderId)
at Microsoft.Exchange.Server.Storage.LazyIndexing.LogicalIndexCache.GetCacheForFolder(Context context, MailboxState mailboxState, ExchangeId folderId)
at Microsoft.Exchange.Server.Storage.LazyIndexing.LogicalIndexCache.GetIndexesInScope(Context context, MailboxState mailboxState, ExchangeId folderId, LogicalIndexType indexType, Column conditionalIndexColumn, Boolean conditionalIndexValue,
SearchCriteria restrictCriteria, SortOrder sortOrder, IList`1 nonKeyColumns, CategorizationInfo categorizationInfo, Table table, Boolean matchingOnly, Boolean existingOnly)
at Microsoft.Exchange.Server.Storage.LogicalDataModel.MessageViewTable.GetViewIndexes(Context context, SearchCriteria findRowCriteria, Boolean matchingOnly)
at Microsoft.Exchange.Server.Storage.LogicalDataModel.MessageViewTable.GetInScopePseudoIndexes(Context context, SearchCriteria findRowCriteria, IList`1& masterIndexes)
at Microsoft.Exchange.Server.Storage.StoreCommonServices.ViewTable.GetFindRowOperator(Context context, Bookmark startBookmark, SearchCriteria findRowCriteria, Boolean backwards, Int32& planCost, Int32& planCardinality)
at Microsoft.Exchange.Server.Storage.StoreCommonServices.ViewTable.FindRow(Context context, SearchCriteria findRowCriteria, Bookmark startBookmark, Boolean backwards)
at Microsoft.Exchange.Server.Storage.LogicalDataModel.MessageViewTable.FindRow(Context context, SearchCriteria findRowCriteria, Bookmark startBookmark, Boolean backwards)
at Microsoft.Exchange.Server.Storage.StoreCommonServices.ViewTable.FindRow(Context context, SearchCriteria criteria, ViewSeekOrigin origin, Byte[] bookmark, Boolean backwards, Boolean& bookmarkPositionChanged)
at Microsoft.Exchange.Protocols.MAPI.MapiViewTableBase.FindRow(MapiContext context, ViewSeekOrigin origin, Byte[] bookmark, Boolean backwards, Restriction restriction, Boolean& bookmarkPositionChanged, Properties& row)
at Microsoft.Exchange.Server.Storage.MapiDisp.RopHandler.FindRow(MapiContext context, MapiViewTableBase view, FindRowFlags flags, Restriction restriction, BookmarkOrigin bookmarkOrigin, Byte[] bookmark, FindRowResultFactory resultFactory)
at Microsoft.Exchange.Server.Storage.MapiDisp.RopHandlerBase.FindRow(IServerObject serverObject, FindRowFlags flags, Restriction restriction, BookmarkOrigin bookmarkOrigin, Byte[] bookmark, FindRowResultFactory resultFactory)
at Microsoft.Exchange.RpcClientAccess.Parser.RopFindRow.InternalExecute(IServerObject serverObject, IRopHandler ropHandler, ArraySegment`1 outputBuffer)
at Microsoft.Exchange.RpcClientAccess.Parser.InputRop.Execute(IConnectionInformation connection, IRopDriver ropDriver, ServerObjectHandleTable handleTable, ArraySegment`1 outputBuffer)
at Microsoft.Exchange.RpcClientAccess.Parser.RopDriver.ExecuteRops(List`1 inputArraySegmentList, ServerObjectHandleTable serverObjectHandleTable, ArraySegment`1 outputBuffer, Int32 outputIndex, Int32 maxOutputSize, Boolean isOutputBufferMaxSize,
Int32& outputSize, AuxiliaryData auxiliaryData, Boolean isFake, Byte[]& fakeOut)
at Microsoft.Exchange.RpcClientAccess.Parser.RopDriver.ExecuteOrBackoff(IList`1 inputBufferArray, ArraySegment`1 outputBuffer, Int32& outputSize, AuxiliaryData auxiliaryData, Boolean isFake, Byte[]& fakeOut)
at Microsoft.Exchange.Server.Storage.MapiDisp.MapiRpc.<>c__DisplayClass29_1.<DoRpc>b__0(MapiContext operationContext, MapiSession& session, Boolean& deregisterSession, AuxiliaryData auxiliaryData) at Microsoft.Exchange.Server.Storage.MapiDisp.MapiRpc.Execute(IExecutionDiagnostics
executionDiagnostics, MapiContext outerContext, String functionName, Boolean isRpc, IntPtr& contextHandle, Boolean tryLockSession, String userDn, IList`1 dataIn, Int32 sizeInMegabytes, ArraySegment`1 auxIn, ArraySegment`1 auxOut, Int32& sizeAuxOut,
ExecuteDelegate executeDelegate)
at Microsoft.Exchange.Server.Storage.MapiDisp.MapiRpc.DoRpc(IExecutionDiagnostics executionDiagnostics, IntPtr& contextHandle, IList`1 ropInArraySegments, ArraySegment`1 ropOut, Int32& sizeRopOut, Boolean internalAccessPrivileges, ArraySegment`1
auxIn, ArraySegment`1 auxOut, Int32& sizeAuxOut, Boolean fakeRequest, Byte[]& fakeOut)
at Microsoft.Exchange.Server.Storage.MapiDisp.PoolRpcServer.EcDoRpc(MapiExecutionDiagnostics executionDiagnostics, IntPtr& sessionHandle, UInt32 flags, UInt32 maximumResponseSize, ArraySegment`1 request, ArraySegment`1 auxiliaryIn, IPoolSessionDoRpcCompletion
completion)
at Microsoft.Exchange.Server.Storage.MapiDisp.PoolRpcServer.EcPoolSessionDoRpc_Unwrapped(MapiExecutionDiagnostics executionDiagnostics, IntPtr contextHandle, UInt32 sessionHandle, UInt32 flags, UInt32 maximumResponseSize, ArraySegment`1 request,
ArraySegment`1 auxiliaryIn, IPoolSessionDoRpcCompletion completion)
at Microsoft.Exchange.Server.Storage.MapiDisp.PoolRpcServer.<>c__DisplayClass48_0.<EcPoolSessionDoRpc>b__0()
at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch[T](Action tryDelegate, GenericFilterDelegate filterDelegate, GenericCatchDelegate catchDelegate, T state)).
Event 2: 1013
The mailbox with mailbox guid dc3ce021-64fb-45f0-a99d-3bbd81fcd7e8 caused a crash or resource outage on database "DB" (e56f5767-199c-40e9-a2f2-98486533f303).
Version: 15.01.1713.001
Description: InvalidOperationException: MESSSCS.Context.OnBeforeTableAccess,MESSLDM.LogicalContext.OnBeforeTableAccess,MESSPAJ.JetConnection.GetOpenTable,MESSPAJ.JetTableOperator.TryOpenJetCursorIfNecessary,MESSPAJ.JetTableOperator.MoveFirst,MESSPAJ.JetDeleteOperator.ExecuteScalar,MESSLILIC.Logi
At '6/5/2019 2:49:42 PM' the copy of database 'DB' on this server was unexpectedly dismounted. The error returned by failover was "There is only one copy of this mailbox database (DB). Automatic recovery is not available.". For more specific information about the failures, consult the event log on the server for other "ExchangeStoreDb" events.
We have also this errors on a DAG!
Event 4: 4999
Watson report about to be sent for process id: 50508, with parameters: E12, c-RTL-AMD64, 15.01.1713.005, M.E.Store.Worker, M.E.S.Storage.StoreCommonServices, M.E.S.S.S.Context.OnBeforeTableAccess, System.InvalidOperationException, 5bd9-dumptidset, 15.01.1713.004.ErrorReportingEnabled: True
Is there someone with a solution here? We are in 3 weeks in contact with Microsoft but do not yet have a solution. We already have more than 18 databases down. For now we have a workaround, moving mailboxes to new databases, but who says this is the right solution.
Problem can arise again, seems to be an Exchange bug.