Purge Client task failed and Upgrade 5.3.2 to 5.10 Upgrade went Rollback

[govanda]

Dear McAfee Enthusiast

a few weeks ago I upgraded the McAfee EPO 5.3.0 to 5.3.2 on my client datacenter.

all went well at EPO version 5.3.2. But, When I try to upgrade to EPO 5.10, there are a few warnings in the pre-upgrade application. for example: 'CheckTableSize'.

7604 2020-02-01 04:05:57 I CHECK NAME: Large database tables
7604 2020-02-01 04:05:57 I Entering 'CheckTableSize'
7604 2020-02-01 04:05:57 I Created the group and set the info
7604 2020-02-01 04:05:57 I
ServerString (null)
7604 2020-02-01 04:05:57 I Opt from lsaopen ppolicy is 0
7604 2020-02-01 04:05:57 I
Setting LSA string initialization
7604 2020-02-01 04:05:57 I Status =0
7604 2020-02-01 04:05:57 I new connection created
7604 2020-02-01 04:05:57 I EPOEvents has 10 records.
7604 2020-02-01 04:05:57 I OrionAuditLog has 2924 records.
7604 2020-02-01 04:05:57 I EPOProductEvents has 2729599 records.
7604 2020-02-01 04:05:57 W EPOProductEvents has 2729599 records.
7604 2020-02-01 04:05:57 I EPORollup_Events has 0 records.
7604 2020-02-01 04:05:57 I EPORollup_ProductEvents has 0 records.
7604 2020-02-01 04:05:57 I OrionSchedulerTaskLogDetail has 5426 records.
7604 2020-02-01 04:05:57 W There are tables with over one million records. Try purging the records, else McAfee ePO upgrade will take time.
7604 2020-02-01 04:05:57 I Connection Closed
7604 2020-02-01 04:05:57 I Exiting 'CheckTableSize', result 3
7604 2020-02-01 04:05:57 I Result : Warning
7604 2020-02-01 04:05:57 I EXITED

I have try deleted the record using "Purge client event, Purge Audit, Etc " but the server task failed at Purge Client Event.

2020-02-11 10:04:56,605 ERROR [scheduler-TaskQueueEngine-thread-1] service.ScheduledTaskManagerImpl - execution of task Purge Old Client Event failed
com.mcafee.orion.core.cmd.CommandException: The server returned an unspecified error.
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1313)
at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:999)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:968)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:945)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:437)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:474)
at com.mcafee.orion.scheduler.chainable.Chain.invokeChain(Chain.java:383)
at com.mcafee.orion.scheduler.chainable.Chain.invoke(Chain.java:64)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runTask(ScheduledTaskManagerImpl.java:1577)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1548)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.runValidatedTaskInvocation(ScheduledTaskManagerImpl.java:1502)
at com.mcafee.orion.scheduler.service.ScheduledTaskManagerImpl.execute(ScheduledTaskManagerImpl.java:1313)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:913)
at com.mcafee.orion.task.queue.TaskQueueEngine.runTask(TaskQueueEngine.java:895)
at com.mcafee.orion.task.queue.TaskQueueEngine.access$1000(TaskQueueEngine.java:50)
at com.mcafee.orion.task.queue.TaskQueueEngine$3.run(TaskQueueEngine.java:864)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: The server returned an unspecified error.
at net.sourceforge.jtds.jdbc.TdsCore.tdsDoneToken(TdsCore.java:3676)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2469)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:668)
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:614)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:573)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate(JtdsPreparedStatement.java:727)
at sun.reflect.GeneratedMethodAccessor198.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.mcafee.orion.core.db.sqlserver.JtdsStatementRetryInvocationHandler.invoke(JtdsStatementRetryInvocationHandler.java:87)
at com.sun.proxy.$Proxy6.executeUpdate(Unknown Source)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
at com.mcafee.orion.core.audit.LogPurgeService.purgeByAge(LogPurgeService.java:296)
at com.mcafee.orion.core.audit.LogPurgeService.purgeByAge(LogPurgeService.java:372)
at com.mcafee.epo.commonevents.cmd.PurgeEventCommand.invoke(PurgeEventCommand.java:310)
at com.mcafee.epo.commonevents.cmd.PurgeProductEventsCommand.invoke(PurgeProductEventsCommand.java:17)
at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:1274)
... 21 more

Then I continue the upgrade process even though the 'CheckTableSize' warning is still on. when the processbar reaches around 90%, the process is rollback and failed to upgrade.

need enlightenment sir. any help would be greatly appreciated

[YashT]

Hello @govanda ,

From the error logs the table EPOProductEvents has 2729599 records.

This needs to be purged else the upgrade will fail.

This article will help you resolve same.

ePolicy Orchestrator 5.x database size warning during upgrade
Technical Articles ID: KB79561

[govanda]

Thank you for your answer YashT,

i have already has the server task which is consist purge old client event, threat event, and rolled-up items. everything can be purged except Client Event. the task is failed. the log already attached above.

what should i do ?  thank you

[Former Member · Accepted solution]

Hello @govanda 

I tried checking the logs and it seems that there is some issue with one of the ePO Database Table and it didn't get successfully rolled back.

1: 16:29:43 EPO5100CALog: epoST.err: The following error occurred while executing this line:
1: 16:29:43 EPO5100CALog: epoST.err: C:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\EPOCore\5.10.0.2428\install.xml:631: The following error occurred while executing this line:
1: 16:29:43 EPO5100CALog: epoST.err: C:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\EPOCore\5.10.0.2428\install.xml:167: java.sql.SQLException: Invalid object name 'EPOTaskTagsMT'.
1: 16:29:43 EPO5100CALog: End runRemoteClentCommand(epoST,...) returnCode=1
InstallShield 16:29:43: CallScriptFunctionFromMsiCA() ends
CustomAction Prod_InstallEpoCore returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
Action ended 16:29:43: InstallFinalize. Return value 3.
MSI (s) (30:E8) [16:29:43:803]: Note: 1: 2265 2: 3: -2147287035

I do believe that you have taken the backup of the ePO Server and ePO Database prior to upgrade.

Please restore your current ePO Database and make sure that EPOTaskTagsMT table is available.

Once the restore is completed successfully please follow KB66616 KB71825 and KB90222
For events please run the ePO 5.3.2 In-built Server Task to purge the events.

You can use https://kc.mcafee.com/corporate/index?page=content&id=KB68961 for removing the events.

While checking the logs:

1: 16:04:31 EPO5100CALog: NOTE: Setup detected that the 8.3 naming convention is not enabled.
https://kc.mcafee.com/corporate/index?page=content&id=KB51431

You can always open a new service request with support to get this towards resolution.

Was my reply helpful?
If this information was helpful in any way, or answered your question, will you please select "Accept as Solution" in my reply, or give kudos as appropriate, so together we can help other members?

[Former Member]

Adding to the previous comment:

Please follow the below KB to use:

Transport Layer Security and cipher suite requirements for ePO 5.10.x

https://kc.mcafee.com/corporate/index?page=content&id=KB91296

[govanda]

okay thank you @Former Member @cdinet  for the answer. i am going to ask my client is there any table EPOTaskTagMT or not.

before i troubleshoot the NamingConversion and TLS/Chipper. i try to run Server task to purge the event but failed. from the logs that i attached, why the task is failed ?  is there any relation to the TLS's Server or EPOTaskTagMT table that is thought to be missing ?

Im afraid that my client wont allowed me to do any SQL script Job, because their SQL Server is joined with another application

[Former Member]

Hello @govanda 

Thanks for your response.

I have checked the logs and able to see below error but that is not explaining in detail might be Debug logs will be helpful.

2020-02-11 10:02:37,656 ERROR [scheduler-TaskQueueEngine-thread-4] service.ScheduledTaskManagerImpl - execution of task Purge Old Client Event failed
com.mcafee.orion.core.cmd.CommandException: The server returned an unspecified error.

 is there any relation to the TLS's Server or EPOTaskTagMT table that is thought to be missing ?

Both are different,

EPOTaskTagMT is related to Client Task Assignment Tag Criteria,

TLS will use to make outbound connection from ePO.
https://kc.mcafee.com/corporate/index?page=content&id=kb90222

https://kc.mcafee.com/corporate/index?page=content&id=KB91296

We need to enable the ciphers as well on ePO and as well as Database, Once the done reboot the servers.

This is a simple tool which IIS Crypto.

If your customer is not comfortable with running tool on the Database due other hosted other applications.

In that scenario we do have a Knowledge base which will help you to move the database from one server to another server.

KB68427

Was my reply helpful?
If this information was helpful in any way, or answered your question, will you please select "Accept as Solution" in my reply, or give kudos as appropriate, so together we can help other members?

[govanda]

wow @Former Member your explanation is so detailed

so, i runned the script (on my own lab) to purge the threat events and its running clear. but its doesnt affect the total row from table dbo.EPOProductEventMT which i have problem with. 

from the report pre-installation auditor, the table dbo.EPOProductEventMT has exceed 1 million that trigger the warning.  i expect running the build-in server task : purge-client event (which is failed)  will delete the row from dbo.EPOProductEventMT.

but in replacement from failed server-task  to run Query KB68961, its doesnt affect that epoproductevent table.

[cdinet]

Try purging in smaller chunks of time to see if it will then purge.  You can also try this (change date accordingly):

delete top (10000) from EPOProductEventsMT
where detectedutc < '2020-01-01'
while (@@rowcount > 0)
begin
delete top (10000) from EPOProductEventsMT
where detectedutc < '2020-01-01'

[govanda]

okay @cdinet thank for the query you give. i will try soon on my client's. i hope there wont be any SQL related error 😄

Thank you so much