Hello All,
I am having some troubles backing up a large database (500GB) using Backup Exec 2010 R3, ultimately the job fails with:
Final error: 0xe00084af - The directory or file was not found, or could not be accessed.
However I believe that the problem is occuring because the consistency check is taking a while to complete and volsnap is deleting the snapshot. Here is an overview of what is happening using one of the failed jobs as an example.
At 1:54:31 PM the job starts and backup to disk media is allocated.
At 1:54:48 PM the following event is logged on the SQL server: I/O is frozen on database IMAGING. No user action is required. However, if I/O is not resumed promptly, you could cancel the backup.
At 1:54:51 PM the following event us logged on the SQL server: I/O was resumed on database EMMS_IMAGING. No user action is required.
At the same time on the SQL server I get confirmation that the SQL server has been backed up: Database backed up. Database: IMAGING, creation date(time): 2013/11/27(19:10:48), pages dumped: 64896058, first LSN: 9042:106279:60, last LSN: 9042:106303:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'BE_SQLAgent-IMAGING__d85e9eaf_68b4_4c61_81ba_9070ab4ac388_'}). This is an informational message only. No user action is required.
At this stage the backup job goes to 'Queued' on the job monitor in Backup Exec.
At 2:08:03 PM event ID 25 from source volsnap logs this entry on the database server: The shadow copies of volume E: were deleted because the shadow copy storage could not grow in time. Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.
E: is the drive that the database file is located and I have checked and I/O load is low on that drive at the time of the snapshot.
At 2:22:05 PM the SQL server logs this: DBCC CHECKDB (IMAGING) WITH physical_only executed by DOMAIN\BackupExec found 0 errors and repaired 0 errors. Elapsed time: 0 hours 27 minutes 13 seconds. Internal database snapshot has split point LSN = 00002352:00019f74:0001 and first LSN = 00002352:00019f73:0001.
At 2:22:06 PM the backup component of the job starts and finishes list the following warning: WARNING: "SVR-SQL-01\IMAGING" is a corrupt file. This file cannot verify.
Then at 2:22:20 PM the job completes with the following: Job ended: Friday, 27 December 2013 at 2:22:20 PM
Completed status: Failed Final error: 0xe00084af - The directory or file was not found, or could not be accessed. Final error category: Job Errors For additional information regarding this error refer to link V-79-57344-33967
I have changed the selection list of the job to only backup a small database on this server and the backup completes successfully and there is no entry from the volsnap source about deleting snapshots.
If I disable the consistency check component of the backup job then the job starts the backup component immediatly and the job completes successfully.
What I think is happening is that the backup job initaites a VSS snapshot on the database server, and then starts the conisitency check part of the backup. However the backup sevrer never tells the database server that it has completed the snapshot, so volsnap times out saying that the storage could not grow in time, and it deletes the snapshot. When the backup job completes the consistency check the snapshot has gone and the backup fails.
Has anybody come across this issue before and is aware of a resolution or a work around?
Thanks
Tim