EMC Backup Procedures Joseph Syu 11/12/2010 group meeting 1.

1

EMC Backup Procedures

Joseph Syu11/12/2010

group meeting

2

Highlights• Networker Backup and Restore• How to check Backup Schedules• How to perform Daily Monitoring• Weekly Clones and Offsite tapes• Eject/Insert tapes in MSL4048• How to deal with 10-year Archives• How to create Encryption keys• How to run Reports• How to login various tools• Documentation• Question

3

EMC Networker Backup and Restore

• How to perform Backup:Go to Networker->Monitor->Groups, select group then right click to “Start”

• How to perform Restore:Restores are performed from client end, sysadmin will initiate jobs and “pull” to their machine (unlike TiBS “push” restores)

4

(re-run backup example)• Right click the selected group then “Start”

5

How to check Backup Schedules

• Go to Networker->Configuration->Schedules tab for backup dates

• Select Group properties for backup time• Current schedules:– M-F daily full (no backup in weekends)

automation start from ~19:00-4:00am nightly– Tuesday night production system also clone tapes

6

(backup schedule example)

7

How to perform Daily Monitoring

• From Networker-> Monitoring->Groups tabThe failed backups are listed on the top with red cross (X) status icon - ignore any disabled groups

• Possible results of investigation:1. False alarm2. Rman pre-processor failure – system or DB fault?3. Rman okay but backup failure – rerun backup only4. Backup okay but weekly cloning failed – reset drives

(possible Networker bug)5. Other failures

8

(monitoring status screen example)

9

(false alarm 1/2)Bssopsf02:/rman/ebsqa appeared to be failed:

10

(false alarm 2/2)

Query Save Set detail and its list showing “browsable” status so okay (live demo if possible)

11

(rman preprocessor failure)EBSDEV_SOUCE

said “Unable to run precintsave…..”

Check with sysadmin for reason of failure, remove the fault then rerun backup

12

(rman okay but backup failed)Go to Client, select proper saveset-> Properties -> Apps &

Modules tab, comment out “savepnpc” from Backup command, rerun backup then restore the command after:

13

(backup did not run)

BSSOAK did not run, should be client end problem

14

Weekly Clones and Offsite tapes• Every Wednesday verify backup and clone

status, rerun if necessary• Current clone tape should give to PREP (Sal) so

Recall picks up on Friday for offsite• Tapes returned (two weeks cycle) should put

back to library if not 10-year archives• If clone jobs are not entirely fixed before Friday,

pull out the original copies (currently two tapes) to offsite, per Mike Woods

15

Eject/Insert tapes in MSL4048

• From Networker, identify current week clone tape barcode and slot

• From MSL4048 command view or console, move tape to empty mailslots

• Go to WH5NW to eject mailslot and eject the tape, insert returned tapes to mailslots if any

• Close mailslot tray• From Networker, label a new tape for next clone as

FINCLONELTO4 (recyclable tapes or older than 30 days)• For returned tapes, move to empty slots then relabel as

FINLTO4POOL for reuse

16

(move tape to mailslot)

17

(label a tape)Tape labels: FINLTO4POOL or FINCLONELTO4 (clone)

18

How to deal with 10-year Archives

• Archives should keep for 10 years offsite (Recall)• PeopleSoft archive tape is the next clone after 3rd

Friday of the month• BSE archive tape is the next clone after 3rd business

day of the month• Determine archives when weekly tapes returned

from Recall, do not reuse them• Fill out Records Storage and Disposition (RSD) form

then signed by Kathryn Duerr , then send offsite by PREP/Recall

19

How to create Encryption keys

• New encryption keys should be created monthly, after BSD archive and before next Tuesday (weekly clone date)

• Unload all tapes from drives• Mark all appentable tapes “full” (read-only)• Create new keys, then backup to token “Green”

with a new password, update LockNotes• Sync new key to DR site (we have no access yet)

20

(unload all tape drives)

• From Networker->Devices, right click drives to unmount

21

(mark appentable tapes full)• Right click media and change mode to Read Only

22

(create new encryption key)

From Command View MSL -> Configuration -> SecurityEnter PIN for token “Green”, then create a new write key by clcking

“Apply” button:

23

(backup token with new password)New password is

required to save token

The “save” button will be available after submit

Download and save token to local disk

Saved token can be restored later in case USB corrupts

24

How to run Reports

25

How to login various tools

• Bss-ts2 or ts3, remote desktop login as joes-admin• Networker Manager console: login as joes

from there to launch finbkp.fnal.gov – Networker Administration

• Command View MSL: https://192.168.200.132/ login as Administrator (password in LockNotes)

• MSL4048 console: in WH5NW, usually do not need to login, otherwise use the same password as Command View

26

Documentation

• Networker/Legato manual in \\afs\fnal.gov\admin\SAN\MIS_SAN

• MIS Backup Transition Logs in\\afs\fnal.gov\admin\SAN\MIS_SAN\Networker

• Procedures\\afs\fnal.gov\admin\SAN\Procedures\EMC-Networker-Backup

27

Questions ?

thanks