Recovery-Oriented Computing User Study Training Materials October 2003
Feb 07, 2016
Recovery-Oriented ComputingUser Study
Training Materials
October 2003
Slide 2
Overview
• Informed consent & Introduction
• User study scenario & your role
• Training (20 minutes)
• Two study sessions (30 minutes each)
• Wrapup and questionnaire
Slide 3
Informed Consent
• Please read the overview of the study and the informed consent form– please feel free to ask any questions you have
about the experiment, its goals, its procedures, etc.
• If you agree to participate in the experiment, please sign the informed consent form
Slide 4
Introduction
• This study is evaluating new recovery tools – the tools are designed to help system
administrators recover from problems affecting server systems
• You will be playing the role of a system administrator– in each of two sessions, you will be trying to
recover an e-mail server system from a pre-existing problem
Slide 5
Introduction (2)
• In each session, you may (or may not) be given an experimental recovery tool to use
• We are trying to understand when the tool is useful for you and when it is not– so if you are given the tool, please think carefully
about whether or not to use it when you are attempting to recover from a problem» at the end of the session, you will be asked to explain
why you chose to use (or not use) the tool
Slide 6
The Scenario
Slide 7
User Study Scenario
• You are one of several system administrators of an electronic mail (e-mail) service– the administrators work in shifts– the study starts when you arrive for your shift
• You arrive to find users complaining that the e-mail service is not working– you will be provided with details of the complaint– the e-mail failure may be caused by:
» failure of the e-mail software, or» an error made by the administrator on the previous shift
Slide 8
User Study Scenario: Your Role• Your responsibilities and goals:– restore the e-mail service to normal operation
as quickly as possible– minimize the amount of lost e-mail and user
work
• Note:– you should prioritize restoring service over
preserving changes made by other administrators
Slide 9
User Study Scenario: Resources• Resources you will have:– a log of all actions performed by administrators
in previous shifts– a day-old backup of the server’s file systems– the Internet– a test e-mail account– a guru
» during each session, you may make up to one request for help to the guru
• Plus any experimental recovery tool that we provide (described later)
Slide 10
Training: E-mail Server
Slide 11
E-mail Overview
• This study concerns e-mail store servers– e-mail stores receive and store e-mail for their users
» users’ mailboxes live on the e-mail store
– they do not handle sending or routing of outgoing mail
• E-mail stores use two protocols– SMTP: used to deliver incoming e-mail to a mailbox
» SMTP is spoken between a remote server that sends the message, and the local recipient e-mail store server
– IMAP: used to retrieve & manipulate mail in a mailbox» IMAP is spoken between a user’s e-mail client and their
local e-mail store server
Slide 12
E-mail Server Configuration
• Mailboxes are text files in /var/mail, e.g. /var/mail/user173• sendmail: process that receives and delivers incoming e-mail• imapd: process that provides remote access to mailboxes• Mail store configuration files can be found in /etc/mail
E-mail Server (Linux)undovmN.cs.berkeley.edu N={1,2,3}
Mailboxes/var/mail/userNNN
SMTPServerProcess
sendmail
IMAPServer
Process
imapdInternet incominge-mail
Users
readinge-mail
SMTP IMAP
Slide 13
Simple Familiarization Task
• Take some time to get familiar with the console and the e-mail system– by performing a basic task as described below
• Goals:– ensure sendmail is running– reconfigure server to recognize mail sent to
[email protected]– restart sendmail to activate reconfiguration
• First step:– connect to undovm3.cs.berkeley.edu with ssh
continues...
Slide 14
Simple Familiarization Task (2)• Next, check if sendmail is running:– execute the command:
ps ax | grep sendmail
• Reconfigure server to accept new host name:– edit /etc/mail/local-host-names to add the line:
roc.cs.berkeley.edu
• Finally, restart sendmail:– run /etc/init.d/sendmail restart
• Try this task now!
Slide 15
Training: Experimental Recovery
Tool
Slide 16
Recovery Tool: an Undo System• The undo system can undo administrative
changes to the e-mail store, including:– changes to configuration files– software upgrades– deleted or altered files
• It can be used to restore the e-mail server to a previously known-good state– by “rewinding” to a date when the system worked OK
• The undo system preserves incoming e-mail and user mailbox changes
Slide 17
When Can the Undo System Help?• The undo system is useful:– when you cannot tell what is causing a problem
» but you know that the system was working at some point in the past
– when a problem affects system state» typically, the same cases where restoring a backup
would fix the problem
• It does not help when the problem does not affect state– like if a server process (e.g., sendmail) has
crashed cleanly without corrupting state
Slide 18
Why Use the Undo System?
• Unlike using a backup, the undo system also repairs the side effects of problems– example: if a problem caused e-mail to be lost,
using undo to fix the problem will restore the lost e-mail» the undo system does this by recording incoming e-mail
and users’ mailbox edits, then restoring them during recovery
• Undo is also useful when you cannot diagnose a problem– simply undo the system to a point in time when it
was known to be working
Slide 19
Undo System Operation
• An undo cycle has two stages:– rewind: the e-mail system’s state is reverted to the
way it appeared at a past time (the “rewind point”)» all changes to the system made since the rewind point
are undone, including:• changes made by administrators• changes due to software bugs• incoming e-mail delivery and user mailbox edits
– commit: makes the rewind permanent but restores incoming e-mail & user mailbox edits to present time
• Net effect: undo cycle undoes all changesexcept incoming e-mail and mailbox edits
Slide 20
Illustration of Undo Cycle
• Before undo:
• After rewind:
• After commit:
time
Rewind point
admin changes
user events(incoming e-mail, mailbox edits)
time
time
admin changes
user events(incoming e-mail, mailbox edits)
admin changes
user events(incoming e-mail, mailbox edits)
user event
admin change
undone changes
restored user events
note that admin changes remain undone
Slide 21
Controls for the Undo System
• Rewind: begins an undo cycle– defines a rewind point and undoes all later changes– may cause e-mail server to automatically reboot– takes 4 to 5 minutes to execute
• Commit: completes the undo cycle– makes the rewind permanent
» restores incoming e-mail & mailbox edits to present time
– takes about 5 minutes to execute
• Cancel: aborts the undo cycle– restores e-mail server to the state it was in before
rewinding
Slide 22
Undo System Interface
• Main window: normal state» time is divided
into 5-minute intervals
» each interval contains user events like incoming mail
» it’s fastest to rewind to a checkpoint
Timeline(color indicatesrelative load)
Current timeCheckpoints
Current undo status
Intervalscontaining
checkpoints
Intervals
Slide 23
Undo System Interface (2)
• Main window: rewound state
Current undo status
Commit andCancel buttons
Current time (inthe past) indicates
undo point
History of undooperations
Slide 24
Undo System Interface (3)• Event window– used to initiate rewind– to view, double-click on an interval in main window
Selected event(rewind point)
Current time
Click to invokeundo cycle
Description of event(here, user170 is examining their mailbox)Event sequence #
Slide 25
Familiarization, Part II
• Try out the undo system interface– note: actually performing an undo cycle may take 10 or more
minutes to complete
• Familiarize yourself with the various resources available to you during the study– Outlook Express e-mail client– the test e-mail account:
[email protected] N={1,2,3}
– the system backup: /backup– books, documentation, the Internet– guru advice: at most one question per session
Slide 26
Resources for More Information• E-mail in general
– About Internet email protocols http://perl.about.com/library/weekly/aa020600a.htm
– E-mail references: http://www.newt.com/email/references.html
• Sendmail– O’Reilly Sendmail book (next to your workstation)– Sendmail home page: http://www.sendmail.org– SMTP RFC: http://www.isi.edu/in-notes/rfc2821.txt
• IMAPd– IMAP general info: http://www.imap.org/– UW-IMAP home page: http://www.washington.edu/imap/– IMAP RFC: http://www.isi.edu/in-notes/rfc3501.txt