☰
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
VietDuc19
Level 10
Report Inappropriate Content
Message 1 of 19
‎01-23-2019 02:55 AM

Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Hi everyone!

Currently, I am using ePO 5.3.3. I created a server task "Purge Server Task Log" to purge records older than x days. When I manually click "Run" to run task, it cannot purge and server task log return status fail. I checked orion.log on ePO server and got the following error

2019-01-23 17:44:38,148 ERROR [scheduler-TaskQueueEngine-thread-22] log.TaskLogPurgeService - SQL error when trying to purge log records: Transaction (Process ID 69) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
java.sql.SQLException: Transaction (Process ID 69) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:373)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2985)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2418)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:614)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:573)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.execute(JtdsPreparedStatement.java:787)
at sun.reflect.GeneratedMethodAccessor625.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.mcafee.orion.core.db.sqlserver.JtdsStatementRetryInvocationHandler.invoke(JtdsStatementRetryInvocationHandler.java:87)
at com.sun.proxy.$Proxy17.execute(Unknown Source)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at com.mcafee.orion.task.log.TaskLogPurgeService.purgeRecordsBasedOnAge(TaskLogPurgeService.java:83)
at com.mcafee.orion.scheduler.command.PurgeTaskLogCmd.invoke(PurgeTaskLogCmd.java:96)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:999)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:968)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:945)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:437)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:474)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:383)
at com.mcafee.orion.scheduler.chainable.Chain.invoke(Chain.java:64)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runTask(ScheduledTaskManagerImpl.java:1577)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1548)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1502)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.execute(ScheduledTaskManagerImpl.java:1313)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:913)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:895)
at com.mcafee.orion.task.queue.TaskQueueEngine.access$1000(TaskQueueEngine.java:50)
at com.mcafee.orion.task.queue.TaskQueueEngine$3.run(TaskQueueEngine.java:864)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I am not got at database design and I don't understand the reason. I hope everyone can help me find out how to fix this error.

Thank you so much

0 Kudos
Reply
1 Solution

Accepted Solutions
cdinet
Employee
Employee
Report Inappropriate Content
Message 11 of 19
‎01-29-2019 04:39 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

No, do not mess with that at all please.  I would highly recommend opening a ticket with McAfee so we can see what is causing the deadlocks.  There are blocking research queries we can run and also look at software versions that may have known issues with it.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

View solution in original post

0 Kudos
Reply
18 Replies
Hem
Employee
Employee
Report Inappropriate Content
Message 2 of 19
‎01-23-2019 03:24 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

I suggest to open case with support. We may ask data as per KB60316 (can be accssed after logging into portal).

0 Kudos
Reply
cdinet
Employee
Employee
Report Inappropriate Content
Message 3 of 19
‎01-23-2019 04:27 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

You might want to check to see how fragmented the database is and run the reindex script that is in KB67184.  I would recommend to do this with all epo services off to start with, then use it in your regular maintenance.  If your db is highly fragmented, that can cause this, though there are other causes.

If that doesn't resolve, then definitely open a case with support.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

0 Kudos
Reply
VietDuc19
Level 10
Report Inappropriate Content
Message 4 of 19
‎01-24-2019 06:48 PM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Dear hem, Dear cdinet

I tried to defragment my ePO database by using script "RebuildReorganizeIndexes-V4.zip" attached on KB67184.

After defragmentation success, I manually run "Purge Server Task Log" again but task stuck at 75% and Duration: 16 hours 13 minutes. I looked into orion log of ePO server and I saw the error:

2019-01-24 17:33:10,554 ERROR [scheduler-InternalTask-thread-2] pip.TelemetryUploader - Could not get orion server properties for Iam CredentialsInvalid input.
2019-01-24 17:33:10,586 ERROR [scheduler-InternalTask-thread-2] pip.TelemetryUploader - Not able to fetch access token, please check if IAM server is reachable

Purger_Server_Task_Log.png

0 Kudos
Reply
VietDuc19
Level 10
Report Inappropriate Content
Message 5 of 19
‎01-25-2019 01:22 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Until now, my purge task finished but it failed. I checked orion.log and got the following error:

 

2019-01-25 13:53:05,044 ERROR [scheduler-TaskQueueEngine-thread-26] log.TaskLogPurgeService - SQL error when trying to purge log records: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_OrionSchedulerTaskLog_OrionSchedulerTaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionSchedulerTaskLogMT", column 'ParentId'.
java.sql.SQLException: The DELETE statement conflicted with the SAME TABLE REFERENCE constraint "FK_OrionSchedulerTaskLog_OrionSchedulerTaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionSchedulerTaskLogMT", column 'ParentId'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:373)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2985)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2418)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:614)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:573)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.execute(JtdsPreparedStatement.java:787)
at sun.reflect.GeneratedMethodAccessor713.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.mcafee.orion.core.db.sqlserver.JtdsStatementRetryInvocationHandler.invoke(JtdsStatementRetryInvocationHandler.java:87)
at com.sun.proxy.$Proxy17.execute(Unknown Source)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
at com.mcafee.orion.task.log.TaskLogPurgeService.purgeRecordsBasedOnAge(TaskLogPurgeService.java:83)
at com.mcafee.orion.scheduler.command.PurgeTaskLogCmd.invoke(PurgeTaskLogCmd.java:96)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:999)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:968)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:945)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:437)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:474)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:383)
at com.mcafee.orion.scheduler.chainable.Chain.invoke(Chain.java:64)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runTask(ScheduledTaskManagerImpl.java:1577)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1548)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1502)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.execute(ScheduledTaskManagerImpl.java:1313)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:913)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:895)
at com.mcafee.orion.task.queue.TaskQueueEngine.access$1000(TaskQueueEngine.java:50)
at com.mcafee.orion.task.queue.TaskQueueEngine$3.run(TaskQueueEngine.java:864)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

0 Kudos
Reply
VietDuc19
Level 10
Report Inappropriate Content
Message 6 of 19
‎01-28-2019 12:16 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

I tried to restart McAfee ePO service (Event Parser; Server and Application Server) and SQL server service. After that, I purge server task log again and it worked. I can purge server task log after x days.

But I don't think this is best solution. I hope anybody can give me the best suggesstion or I hope this can be fixed on the next update if this is a bug.

0 Kudos
Reply
cdinet
Employee
Employee
Report Inappropriate Content
Message 7 of 19
‎01-28-2019 05:52 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

Check KB's 84114 and 83384 to ensure the dbcleanup task is running properly.  That can cause all kind of issues and a restart of services temporarily clears things up.  The issues in those kb's can still occur.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

0 Kudos
Reply
VietDuc19
Level 10
Report Inappropriate Content
Message 8 of 19
‎01-28-2019 07:35 PM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution
  •  Following the KB84114: 

- When I running the second query: SELECT * FROM OrionTaskQueueMT WHERE TaskDescription LIKE '%DBCleanUp%'

The return result look like this:

Second_Query.png

- The first query: SELECT Id, UniqueKey, Enabled FROM OrionSchedulerTaskMT WHERE UniqueKey = 'DbCleanupTask'

Return the result:

First_Query.png

I think everything are correct. 

0 Kudos
Reply
VietDuc19
Level 10
Report Inappropriate Content
Message 9 of 19
‎01-28-2019 08:02 PM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

You are correct. Restart services is only temporary solution. I run purge task log and it's failed again. Orion.log on ePO server still show the error:

2019-01-29 10:57:59,554 ERROR [scheduler-TaskQueueEngine-thread-14] log.TaskLogPurgeService - SQL error when trying to purge log records: The DELETE statement conflicted with the REFERENCE constraint "FK_OrionTaskQueue_TaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionTaskQueueMT", column 'TaskLogID'.
java.sql.SQLException: The DELETE statement conflicted with the REFERENCE constraint "FK_OrionTaskQueue_TaskLog". The conflict occurred in database "EPO-MCAFEE", table "dbo.OrionTaskQueueMT", column 'TaskLogID'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:373)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2985)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2418)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)

0 Kudos
Reply
VietDuc19
Level 10
Report Inappropriate Content
Message 10 of 19
‎01-29-2019 12:10 AM

Re: Transaction was deadlocked when running client task "Purge Server Task Log"

Jump to solution

When I run query: SELECT * FROM OrionTaskQueueMT

I got the following result:

Query.png

There is a task with NodeUUID and TaskLogID are not equal NULL. Should I update that values to NULL?

0 Kudos
Reply
You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use our Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from product experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by employees.
Join the Community
Join the Community

TrellixSkyhigh Security | Support Trellix.com SkyhighSecurity.com

Legal Privacy Copyright © 2023 Musarubra US LLC