-
THE MYSTERIOUS OUTLOOK OUTBOX
Understand why messages get stuck in the Outbox folder in an
Exchange environment.
(Today's post is courtesy of Scott Bradley, Principal Escalation
Engineer, Office Technical Support.)
A TRIP BACK IN TIMETHE EVOLUTION OF OUTLOOK
A thorough understanding of mail submission/delivery starts with
a trip back in time. Years ago, before
Outlook was born, Microsoft defined some standards for messaging
applications. These standards are
commonly referred to as MAPI (Messaging Application Programming
Interface), and detailed documentation
can still be found on MSDN. At the root of the MAPI world is
this idea of three main messaging components:
1. A place to actually STORE the messagesthe MAPI MESSAGE
STORE
2. A method to SEND the messagesthe MAPI TRANSPORT
3. A place to store and look up ADDRESSESthe MAPI ADDRESS
BOOK
Most people are familiar with the common stores and address
books in the Exchange/Outlook world. We have
the mailbox and Global Address List (GAL) from Exchange, and
things like the PST/OST file and Offline Address
Book or Contacts Folder (which is abstracted into an address
book by the Outlook Address Book service) from
the Outlook side. But this middle part of MAPI, the transport
layer, is less understood.
In non-Exchange scenarios, the transports are much easier to
understand. Consider the POP/SMTP setup. You
use the POP protocol to receive mail, and SMTP to send mail. The
protocols are easy to read and understand.
You can look at a network trace and see the commands and the
mail going over the wire, etc. In the
Exchange scenario, things are more complicated.
Along with this notion of the MAPI TRANSPORT being responsible
for actually sending mail, MAPI includes the
idea of a spooler which is something that manages the
transports. It loads them up, checks for mail that
needs to be sent and actually calls into the transports code to
send the mail. You can think of it as the driver
of mail delivery. Until Outlook 2002, the spooler was
implemented in a completely separate executable. So
when you ran Outlook (or any other MAPI application), you would
also see a process named MAPISP32.EXE in
your task list. The spooler ran independently of Outlook. When
it came time to send mail, MAPISP32.EXE woke
up and did the work of driving the mail.
In Outlook 2002 the design changed, and the spooler code was
moved into the Outlook executable itself. This
design still exists today and has some interesting implications.
Understanding these implications means you
really have to embrace the fact that Outlook IS the spooler.
Anything that you expected to happen with the
spooler, now only happens when Outlook is running.
-
Its easy to see the result of this design when you send mail
from another application when Outlook is not
running. Consider this scenario:
1. Outlook is installed but not running on your computer.
2. On the desktop, you right-click a file and choose Send to
Mail Recipient.
3. Outlook comes to life and gives you a message to work on.
4. You type a recipient name and a message, and hit Send.
5. Later you shut down your computer and go home.
6. The next morning, the recipient is mad because they never got
the mail.
What happens in this case is that immediately AFTER you hit send
in step 4, Outlook is done, so it closes. Now
Outlook is not running, and therefore no spooler is running, and
so nothing is going to send your mail. The
next time you launch Outlook, you will see the mail in the
Outbox. Assuming Outlook stays open long enough
for a spooling cycle to happen (default is 15 minutes), the mail
will be sent successfully.
One easy way to conceptualize the spooler in Outlook is to think
of it as everything in the Send Receive
Groups dialog. This dialog roughly maps to the things the
Spooler code does and how Outlook manages all
the options.
As an aside, look closely at the screenshot on the right. This
is from a standard cached mode configuration of
Outlook. You will notice that almost all the options are gray
and unavailable. This is because the Exchange
account is NOT included in this group in a standard cached mode
configuration. Note the red X over the
account on the left, and the empty checkbox at the top. In
cached mode, you do NOT typically want the
send/receive group settings to apply. Cached mode has its own
logic for synchronizing and sending mail, and it
is notification based, not scheduled like the send/receive
groups.
Finally, consider the impact of the cached mode feature that
came along for Outlook 2003. Cached mode is
many things, but one specific part of the cached mode design is
that mail delivery uses this spooler process in
-
a more intricate way, and uses it for ALL cached mode sends.
Again, it works independent of the settings in the
Send/Receive Groups, and it has switches and configuration
choices that add complexity.
So there are four key takeaways to start our understanding of
Outbox problems:
1. MAPI Transports are the way that Outlook sends mail
2. There is this notion of a spooler that is the key to managing
and driving the sending mail cycle
3. Outlook.EXE *IS* the spooler ever since Outlook 2002
4. The cache mode feature drastically changed the importance of
the spooler
MAIL DELIVERY IN EXCHANGE
Up until now, I have described the general MAPI ideas around
mail delivery and how Outlook implements
these options. Now we need to understand some specifics in the
Exchange Server configurations. Again, some
history is needed to help understand how we got to where we are
now.
In early versions of Outlook/Exchange, users almost always used
Outlook in the online mode of operation.
You interacted with Mail and Addresses directly from the
Exchange Mailbox. This is in contrast to todays world
where most people use cached mode. In cached mode, you work with
data and addresses from a LOCAL data
store (your OST file), and then synchronize your data to the
mailbox. Note that SENDING mail is not the same
as synchronizing. The Outbox folder is never part of
synchronization. Mail submission and delivery is different
from synchronizing.
There is a fundamental difference in mail submission/delivery
between online and cached mode.
Understanding this difference is one big key in your ability to
grasp the entire Outbox story. For this blog entry,
I am going to define two different methods to deliver mail:
1. Transport Send (this is the spooler method that I described
in detail above)
2. Store Send (this is the second method that ONLY is used in
the Exchange Online scenario)
Since we know that in cached mode you work from your OST and not
online, you can see that all cached mode
mail delivery must happen using the Transport Send method. If
you are running in the old online profile (which
is common even today on terminal server configurations), then
mail delivery is typically done using the Store
Send method.
So why do we care which method is used? Mainly because a Store
Send does NOT require any of the spooler
process to happen. When the Store Send method is used, the
message is delivered by the server with no further
client interaction. In the send to mail recipient scenario that
I described in the previous section, if the Outlook
profile is configured for online mode, that mail *IS* sent
successfully. Since no spooler is needed to deliver the
mail, it is delivered immediately by Exchange, whether Outlook
is running or not. There are other differences
under the hood for each method. Appendix C has a table that
summarizes the differences.
-
HOW OUTLOOK CHOOSES WHICH DELIVERY METHOD TO USE
A good way to conceptually think about the Outlook decision
process is to think in terms of the Outbox folder
itself. When a message is created in the Outbox folder of the
OST/PST, Outlook and MAPI hook up all the
plumbing so that mail submission happens through the Transport
Send method. When a message is created in
the Outbox folder of the online mailbox, Outlook and MAPI hook
up the plumbing so that mail submission
happens through the Store Send method. So using this concept, if
you are in cached mode you should expect a
Transport Send, and if you are in online mode, you should expect
a Store Send.
Additionally, even if you are in ONLINE mode and would typically
be using the Store Send method, there are
options and configurations that will prevent the use of the
Store Send. The most common example is the
presence of an additional transport. So for example, if you have
an online configuration and you ADD an
SMTP/POP account, you are no longer eligible for delivery via
the Store Send method. Likewise if you add a
meeting service transport like the LiveMeeting service, you must
now send mail using the Transport Send
method. Any additional transport will turn off the ability to do
Store Send with Exchange. Finally there are a
couple of Outlook configuration options that will turn off the
Store Send in online mode. The primary one is
the option to save replies with their original mail. So to
summarize, you can only take advantage of the Store
Send method if:
1. You have Exchange Server.
2. You are configured for online mode and not cached mode.
3. You have no other transports in the profile.
4. You are not using the save replies with original option.
The formal term for the save replies with original is this
setting in the options dialog:
-
THE NEEDS_SPOOLER FLAG
This option gives a good example of an important extra trigger
in the mail delivery process. Even if you are
using the Store Send method, where the spooler (Outlook) is not
required, there are scenarios where the mail
will require the spooler. The save replies with original feature
requires some processing by the spooler. So
even in a Store Send scenario (online mode, no extra transports,
etc.), Outlook will add a special flag to the
message during submission that tells Exchange not to deliver it
immediately. This flag is called the
NEEDS_SPOOLER flag.
The process looks like this from a non-technical point of
view:
1. Outlooks logic runs through some tests and determines that
this message will need the spooler for
some reason.
2. When Exchange accepts the message for submission, it looks
for the this message NEEDS the spooler
trigger.
3. If there is no trigger, then Exchange sends the mail itself
and you have a Store Send.
4. If the trigger is there that says this message NEEDS the
spooler, then Exchange does NOT deliver the
message and waits on another command from the spooler before
sending.
5. When the spooler cycle happens, it sends the message up to
Exchange to Send the message and you
have a Transport Send.
So all the criteria are tested for each method and in the end,
each message either gets the trigger for needs
the spooler added or not.
Notice that if Outlook calculates that a Transport Send is
needed, there is some amount of time between
message submission and message delivery. In a Store Send, there
is only the one submission, but in the
Transport Send, you have the message submission, and then at
some later time, the command to do the
Transport Send coming from Outlook itself. There is a choice in
the Outlook options screen called Send
Immediately when connected and the option is on by default. When
this option is on, Outlook will kick of a
spooler cycle and do a Transport Send immediately after sending
the message if a spooler cycle is needed. If
the option to Send Immediately when connected is turned off,
then the message will only be sent according
to the defined send/receive settings. Since the send/receive
settings, by default, do not include the Exchange
Account for cached mode, it can be pretty easy to have a
send/receive definition that you do not expect. So its
important to examine the send/receive settings and make sure the
expected definitions exist for how often you
want to send mail. Below is the screenshot showing the Send
Immediately option.
-
As a summary of this section, here is the way Outlook thinks
about how a message gets submitted and
delivered:
1. Is this new message I am sending based in an online Outbox
folder or an OST/PST folder? If its online,
route the submission through the Exchange Store Send code. If it
is OST/PST, route the submission
though the Transport Send code.
2. After submitting the message, do a bunch of checks to see if
either this is a straight Transport Send, or
one of the special case Store Sends that need the spooler. If
either of these is true, then kick off a
spooler cycle to do the Transport Send.
3. If this was an OST/PST based send, then do the work to move
the item from the Outbox folder to the
Sent Items folder, and send any NDRs that have occurred.
SUBMISSION STATE
During the time that the message is submitted, but not
delivered, it is in an unusual state. The submitted
state shows up in a few ways. The message in the Outbox is
rendered in italics. Also, you cannot delete the
message in the normal way by simply deleting it from the UI.
Being in the submitted state is special. More
importantly, since there is some amount of time between being
submitted and being delivered, that is an
opportunity for something to abort the submission for the
message. This happens sometimes from add-ins and
external MAPI programs. Sometimes these other components are
monitoring the Outbox folder for items to act
on and they grab the Outbox message and abort the submission
accidently. If something aborts the
submission, the message stays in the Outbox folder until
something else submits it again. In the
-
troubleshooting section, there is information about the
MFCMAPI.EXE tool. This tool can be used to examine
the submission state flags, abort a submission, or resubmit a
message. So it is a good tool for learning about
the submission state.
THE OUTBOX
Next in our quest to understand this topic are some ideas and
concepts around the Outbox folder. From a base
level concept, it is the folder in which new messages are
created. This creates an interesting design when auto-
save kicks in (or when you manually save an unsent message),
because when you do this, the message is
SAVED to the drafts folder. One important fact to consider is
that there is a fundamental difference between a
message being delivered, and the removal of that message from
the Outbox folder. They are two different
things, and they are not atomic (its perfectly fine for one to
happen and not the other). For this reason, one of
the first questions you always want to answer when
troubleshooting stuck in the Outbox folder issues is
whether the recipient actually RECEIVED the email. Think of the
complete process as two steps:
1. The work to transmit the message to the recipient
2. The copying or deletion of the item in the Outbox to
somewhere else
For step 2, most often the message is moved to the Sent Items
folder, but you can specify an option to not
save the sent item, and in the working scenario, the sent
message is just deleted from the Outbox folder. Again
these two steps are completely different code paths in Outlook,
and it is not extremely rare for one to work and
the other to fail. So for example, if the recipient successfully
receives the message, but it still shows present in
the Outbox folder, that means step 2 only failed and you should
troubleshoot those issues, not the delivery
code.
One of the main reasons for the detailed description of Store
Send and Transport Send above was because
that process defines how the removal from the Outbox happens. In
the Store Send case, the Exchange Server
does the removal from the Outbox folder and the move to the Sent
Items folder on the server side. Outlook
need not be running for that to happen successfully, and Outlook
typically cannot cause or fix problems related
to the Exchange work here. If youre in the Store Send scenario,
and mails are stuck in the Outbox folder, the
best place to start troubleshooting is on the server side. On
the other hand, if you are in the more typical
Transport Send configuration, Outlook (the spooler) does the
work of removing the outgoing item from the
Outbox folder and moving a copy to the Sent Items folder.
Problems with items being delivered but not
removed from the Outbox folder in the Transport Send
configuration are typically client side issues and
should be investigated at the Outlook level.
SYNCHRONIZATION
Continuing our investigation of the Outbox folder, our next
peculiarity is that this folder is not part of the
synchronization engine. Consider the scenario where you do not
SEND mail, but you just drag and drop 10
messages into your Outbox in cached mode Outlook. If this were
any other folder, you would expect in a few
-
seconds those messages would be synchronized to the mailbox, and
the folder in the mailbox would have 10
new messages. If you do that with the Outbox folder, you never
get 10 messages in the server Outbox folder.
The Outbox folder is not part of sync except in one small corner
case. To understand that corner case, we must
consider a feature added to Outlook 2007 that is referred to as
the sendone feature.
In versions of Outlook before 2007, when you sent a message, the
message would actually be transmitted over
the wire twice; one time during message submission, and then
again when the item was synchronized up from
the Sent Items folder. Consider this scenario:
1. You create a mail message and attach a significant file,
perhaps a 10 MB video file.
2. You send the message. 10.01 MB goes over the wire to the
Exchange Server for submission.
3. Outlook sends the send the mail now command to Exchange and
then moves the item from Outbox
to Sent Items.
4. The sync engine kicks in and sees a new 10.01 MB message in
the sent item and synchronizes it up to
the mailbox.
Now you have sent 20.02 MB of data just to send the message.
To solve this problem, Outlook and Exchange agreed on a clever
trick to allow the two to work together and
not have to send the message over the wire twice. This is the
sendone feature. It works like this:
1. During submission, Outlook stamps a special property on the
message with information that tells
Exchange how to make a copy of the message on the server
side.
2. Exchange, after receiving the message for submission, makes a
copy (conceptually in the Outbox server
side).
3. Outlook marks the new message that you just created as NOT
NEW. This tells the sync engine to not
blindly sync it up to the server.
4. Outlook then moves the message to the Sent Items folder and
the sync engine sends up the command
that says this message has moved to the Sent Items folder.
5. Exchange gets the sync and since it has that special copy of
the message in its Outbox, it moves the
message server side to the Sent Items folder.
In this way, the sent item is never sent over the wire.
This is a clever design and saves a LOT of network bandwidth,
but you can see there are a couple of places
where things can go wrong. If Outlook tells Exchange in step 1
the wrong information on how to make a copy
of the message, then later when the move sync happens, Exchange
will not find a matching message and be
unable to move the item server side from Outbox to Sent Items.
This will show up as stuck in the Outbox
folder. Likewise, if you have a failure in the sync process in
step 4, the server will not know to move the item
server side. In these cases, the message is stuck in the Outbox
folder when looking online, but looks fine in
the OST folder in cached mode. Finally, you have to sync at
least once for this to work at all. It is possible to
send an email, and then exit very quickly before the sync in
step 4 happens. In that case, since no sync
-
happened, the server leaves the copy of the message in the
server Outbox folder. The next time you sync
successfully everything works, but until you sync once the
message stays in the Outbox folder. Consider this
relatively common scenario:
You have a machine at home running cached mode and a machine at
work running online mode.
As you leave for work, you send a quick piece of mail and exit
Outlook before the sync of the Sent Items
happens.
At this point the mail has been delivered, but server side is
still in the Outbox folder.
You get to work and notice a message stuck in the Outbox folder.
It just never goes away.
Finally 4 days later you happen to be working at home again and
use Outlook long enough for a sync
to happen.
The next day at work you see that the Outbox folder is empty.
The sync has successfully sent the
message to Exchange to move the item to Sent Items in the
mailbox.
Or another common scenario:
You want to keep any mail you send with budget in the subject in
a special folder called Budget
Audit.
Every time you send a mail like this, immediately after sending,
you go and manually copy the message
to the Budget Audit folder before the sync engine kicks in.
Now there is no sent item to force the sendone sync to happen
the right way and the copy of the
message is left in the Outbox folder on the server.
As part of this design, one corner case was added to the
synchronization of the Outbox folder. As was
mentioned earlier, in general the Outbox folder is NOT ever part
of sync. The one exception is for delete
operations. If you have a rogue or unexpected item in the OST
Outbox folder, and you manually delete the
item, then that will delete the item in the server mailbox as
well. This is just a corner case to try to make the
Outbox folder behave with more consistency.
Again, in all cases where the move to Sent Items fails, the
message actually *IS* delivered. So message stuck in
the Outbox folder does NOT necessarily mean the message was not
delivered.
THE SPOOLER CYCLE
Now that we understand the importance of the spooler to mail
delivery, here are some components that help
us understand that process in greater detail. Each Account in
your mail configuration has an associated
message store (for Exchange it is the mailbox for online mode,
the OST for cached mode, if you have a
POP/SMTP account it might be a PST file, etc.). Each message
store keeps track of Messages in the Outbox in
a place internally referred to as the Outgoing Messages Queue.
When the spooler cycle starts, it asks each
account to look in its corresponding message store and gather up
the list of outgoing messages. Then it goes
-
through the messages and submits them to the appropriate
transport for delivery. In considering this design,
you can see that if there are any problems interacting with the
message store, they might manifest as mail
delivery problems. Consider this example:
1. The spooling cycle starts and asks each account for their
outgoing messages.
2. One account says that its using D:\DATA\TESTONE.OST for its
message store.
3. Something has deleted testone.ost from the d:\data
folder.
In this case, the attempt to gather up the outgoing messages for
that account will fail. Interestingly, having
ONE message store give an error does not show up as an overt
error in Outlook. The spooler will just continue
to look at other accounts/stores and if they work and have
messages to send, it will seem like the mail delivery
process worked. So it can be hard to see an error when the error
happens during the outgoing messages
collection.
The next part of understanding the big picture of successfully
delivering an email is to understand another
feature provided by MAPI. The developers of MAPI built in a way
for MAPI applications to do work during idle
time on the computer. Idle time essentially means that you are
not typing or moving the mouse around. Now
computer time is not as expensive as real time, and there are
lots of things that the computer can do with
pretty small amounts of idle time. So we are not talking about
having to have your hands off the computer for
an hour at a time. We are talking about less than a second of
idle time sometimes to give MAPI a chance to do
some work. So whats the big deal about this?
The automatic spooling cycles are all idle time tasks. In almost
all instances, when something thinks its time to
do a spooler cycle (for example, Cached Mode and the Send
immediately option, you pressing the
Send/Receive button, a normal 15 minute Send/Receive cycle
happening, etc.), the spooling cycle is added to
the idle tasks queue. This means for the actual processing of
spooling to happen, you MUST have some idle
time on the machine. As was mentioned before, idle time slices
are very small, and you almost ALWAYS have
plenty of them in normal operations of the computer. In fact
MANY different tasks run during idle time, not just
the spooling cycle. That said, the use of the MAPI idle feature
to drive the spooling cycle leads to two very real
possible problems:
1. If any of the OTHER idle time tasks have a problem, they can
clog up the idle queue and cause anything
waiting to run during idle to not run.
2. System components like keyboard and mouse drivers can prevent
the machine from ever getting idle
time.
Consider this example:
You have a mouse driver for your computer that is an advanced
driver and tries to add a feature set to
make your mouse better in some way.
Even when you dont move the mouse, and your hands are off the
computer, the mouse driver sends a
mouse status message to Windows every .0001 seconds.
MAPI now thinks you never have any idle time. It seems as if the
mouse is ALWAYS in USE.
The spooler cycle may not ever start and mail is not delivered.
Incidentally, this is a very REAL cause of a
failure to download the Offline Address Book, which is another
MAPI idle time task.
-
ADDRESS TYPES
The next topic to cover for a full understanding of the mail
delivery process is the notion of address types.
Historically, these were more important in the design of email.
Today with the proliferation of Exchange and
Internet Mail, almost all of the other types are seen less
commonly. Each email address that you send to is
composed of two different pieces of data in the MAPI world, the
email address and the email address type.
Consider these examples:
Display Name: John Doe
Email Address: [email protected]
Email Address Type: SMTP
Display Name: Jane Doe
Email Address: /O=Contoso/OU=North/cn=Recipients/cn=12345
Email Address Type: EX
A little-remembered fact in MAPI is that when you create an
email, you can use the canonical form to specify
any address type you want. For example, if entered on the To
line:
[mytype:first|last|server|store]
Is a recipient that would be treated like this:
Email Address: first|last|server|store
Email Address Type: mytype
Now by specifying an address of mytype, that means you must have
a transport that will deliver messages to
recipients of mytype. A more common example of the additional
address types would be seen when using a
FAX transport. If you have a MAPI transport installed that will
take your message and render/deliver it as a FAX,
then it registers itself as handler for FAX address types.
Therefore when you address a message like this:
To: [FAX:5551212]
The FAX transport understands this address type and delivers the
mail (which in this case means rasterizing it
into an image and sending the image over the phone, most
likely).
Now lets talk about how this fits into the rest of the big
picture. When Outlook logs onto the Exchange Server,
some configuration is traded between the two. One thing that
Exchange sends down to Outlook is the list of
address types for delivery to be completed. There are almost
always at least three address types in the basic
list:
EX
SMTP
X400
Whatever is in the list is what you can deliver mail to using
the Exchange transport. Note that you might have
another transport service installed that can deliver to other
types. So for example if you have a third party fax
transport installed, you might still use it to successfully send
an email to [fax:5551212]. The list that comes from
-
the Exchange Server is just the list that Exchange knows how to
handle. So again we have to think how that
might show up in a problem scenario. Consider this scenario:
1. Outlook connects to Exchange and asks for the list of address
types.
2. Due to a glitch on the Exchange Server, the list comes back
empty.
3. Now if you try to do a Transport Send in Outlook, it will
always fail. Outlook believes that the Exchange
Server cannot deliver to any address types.
It is not easy to see the address types listed anywhere in
Outlook, but there is one troubleshooting log that will
give an indication that things are probably ok in this area and
we will cover that log in detail in the
troubleshooting section.
NON_DELIVERY REPORTS (NDRS)
We discussed earlier the notion of the two different fundamental
send methods. One important difference to
understand is how NDRs are generated in each scenario. For the
Store Send method, where Outlook and the
spooler are NOT involved, NDRs are generated by the Exchange
Server. For the Transport Send, NDRs are often
generated by the spooler, and as we know, Outlook is the
spooler. The point remains, if you get an NDR from a
Store Send, you must troubleshoot the Exchange side of the
process. If you get and NDR from a Transport
Send, then something client side can be contributing to the
problem.
TROUBLESHOOTING/ISSUES
This section contains information about troubleshooting and will
help your understanding of mail
submission/delivery problems.
TRANSPORT LOGGING
The entire spooling process is logged to a troubleshooting log
named OPMLog.log. Logging is enabled using
the standard UI option in the Advanced pane of the main Outlook
Options dialog:
-
This log is created in the Outlook Logging folder which is a
folder under the system temp folder. The log
contains detailed information about the spooling/transport
process. Note that this logging is not limited to
MAPI/Exchange Server logging. If you have an Outlook profile
with only an SMTP/POP transport, that traffic will
be logged in this file as well. Any mail transport that sends
mail will show up in this logging. The logging will
include among other things:
The subject of the mail being sent
The attempt to logon to the transport
Any errors received during logon or transmission of the
message
In the Exchange case, the number of address types to which
Exchange can deliver mail
In the POP/SMTP case, many details about the download/sending
conversation
Appendix A has an example OPMLog.log file with annotations on
the important pieces of information in the
file.
SYNCHRONIZATION ISSUES FOLDER MESSAGES
The Synchronization Issues folder typically only has messages
when something went WRONG during sync. If no
errors were encountered, no message is created in the folder.
And yet to troubleshoot the sendone process,
we need to know when the Sent Items folder successfully
synchronizes a message. Outlook provides a registry
value to save sync messages in the Sync Issues folder EVEN IF
THERE ARE NO ERRORS. The registry value is
named AlwaysSaveSyncLog and the key is
HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Outlook\OST.
-
WARNING: Do not leave this registry value enabled for any length
of time. It will create a message in the Sync
Issues folder EVERYTIME a successfully sync cycle happens, which
is very often in the cached mode
configuration. This will generate a lot of noise messages and
people will complain if you forget to turn it off.
With the option set to always save the sync logs, you will be
able to check for the Sent Items sync and confirm
whether it is happening, and you will see that items are being
added to the online store. Appendix B has
examples of what the sync issues messages look like in these
scenarios.
REGISTRY SETTINGS
There are several other registry settings that affect the mail
submission/delivery process in a substantial way.
Here is a table with the registry value and a comment on how
they are related to Outbox folder issues:
Registry Value Comment
Key:
HKCU\Software\Microsoft\Office\14.0\Outlook\Options\Mail
Value: Send Mail Immediately
As mentioned earlier, cached mode does not use the
Send/Receive settings to schedule mail delivery. That means
that without something special, when you send a message,
nothing would start the spooling cycle. This value is the same
as
the UI option for Send Immediately When Connected and tells
Outlook in cached mode to start a spooler cycle immediately
after you hit Send. If this value is set to 0, mail sends
wont
happen until a later time and seem to be stuck in the Outbox
folder.
HKCU \Software\Microsoft\Office\14.0\Outlook\Preferences
DelegateSentItemsStyle
This is a tricky setting. There is no UI choice for it, so it is
only
set via the registry setting. To understand the setting, we
need
understand what a delegate is. For this explanation, a
delegate
is a user that has permission to send on behalf of another
user
AND has permission to the other users folders. We will use
the
word Owner to represent the other user. So the delegate has
ability to send mail on behalf of the owner. This value
tells
Outlook whether to move the sent item to the delegates Sent
Items folder or to the owners Sent Items folder.
This is a big deal under the hood because Outlook has to be
able to open the mailbox to get to the Sent Items of the
owner.
It takes a specific set of permissions, and can have
complexity
depending on whether you are in cached or online mode.
It also affects the mail sending process because during the
gathering of the messages to be sent, Outlook must do some
work related to this delegate/owner configuration. If
anything
goes wrong during that work, the mail submission/delivery
will
show some kind of failure.
Key:
HKCU\Software\Microsoft\Office\14.0\Common\MailSettings
Value: StrictAccountOrder
Very rarely used legacy registry key that will prevent
Outlook
from ever doing a Store Send. Even if you meet all the other
requirements such as online mode, no additional transports,
etc., if you have this registry value, you force a Transport
Send.
-
MFCMAPI (RECIPIENT PROPERTIES, PROFILE SECTIONS, SUBMIT FLAGS
AND ABORTSUBMIT,
ETC)
My coworker and MAPI guru, Stephen Griffin, maintains the
definitive MAPI troubleshooting tool. The formal
name is Microsoft MAPI Editor. For historical reasons, many
people just call it by the executable name,
MFCMAPI. The latest version is always available at the main
download location. Below is a screenshot showing
the use of MFCMAPI to interrogate recipient properties for a
messages recipient. I am showing in the property
pane that we are looking at an Exchange type.
Not only does MFCMAPI allow for complete interrogation of MAPI
stuff, Stephen has integrated parsers for
many of Outlook and MAPIs binary structures. So for example, you
can use MFCMAPI to dump the contents of
the binary nickname cache, which is data that might affect
addressing and sending mail. The uses for MFMAPI
in terms of troubleshooting Outbox folder issues are many. For
example:
List and interrogate properties of the MAPI Transport
Services
List and view/change recipient properties
Submit or abort the submission of a message
Check the message flags to see if it is in the submitted
state
-
OTHER RESOURCES
KB Article describing the SendOne Feature
KB Article describing DelegateSentItemsStyle
KB Article describing gotchas with the
DelegateSentItemsStyle
Blog Post with info on fixing stuck messages including issues
with Icloud Transport
KB Article describing the StrictAccountOrder Setting
APPENDIX A SAMPLE OPMLOG.LOG FILE
Log Entry Comment
2013.03.06 10:16:36 >
2013.03.06 10:16:36 HELPER::Initialize called
2013.03.06 10:16:36 Initializing: Finding a Transport
2013.03.06 10:16:36 MAPI XP Call: XPProviderInit in EMSMDB.DLL,
hr = 0x00000000
2013.03.06 10:16:36 MAPI XP Call: TransportLogon, hr =
0x8004011d This is a normal error and should be ignored.
This is because the first logon attempt is done
with anonymous credentials which do not work.
2013.03.06 10:16:36 MAPI XP Call: Shutdown, hr = 0x00000000
2013.03.06 10:16:36 MAPI XP Call: XPProviderInit in EMSMDB.DLL,
hr = 0x00000000
2013.03.06 10:16:36 MAPI Status: (-- -- ---/--- -- ---)
2013.03.06 10:16:36 MAPI XP Call: TransportLogon, hr =
0x00000000 Here the logon succeeds and the Mapi
Transport is ready to be used.
2013.03.06 10:16:36 Initializing: Found a transport, Error code
= 0x00000000
2013.03.06 10:16:36 MAPI XP Call: AddressTypes, hr = 0x00000000,
cAddrs = 3, cUids = 1 Notice the Count of Address here. If this was
0,
all Mail submissions would fail. This shows the
default set of 3 address types (EX, SMTP, X400)
2013.03.06 10:16:36 MAPI XP Call: RegisterOptions, hr =
0x00000000, cOptions = 2
2013.03.06 10:16:36 MAPI Status: (IN -- ---/OUT -- ---)
2013.03.06 10:16:36 MAPI XP Call:
TransportNotify(BEGIN_IN|BEGIN_OUT), hr = 0x00000000
2013.03.06 10:16:36 HELPER::Initialize done, Error code =
0x00000000
2013.03.06 10:16:36 HELPER::GetCapabilities called, Error code =
0x00000000
2013.03.06 11:03:35 Microsoft Exchange: Synch operation started
(flags = 00000001)
2013.03.06 11:03:35 Microsoft Exchange: UploadItems: 1 messages
to send Indication that there ARE messages to send.
Conceptually, the process of getting outgoing
messages has happened and this is the result.
2013.03.06 11:03:35 EXECUTING Put MAPI TASK
2013.03.06 11:03:35 Starting the Spooling Cycle Spooler Cycle is
Happening
2013.03.06 11:03:35 MAPI Status: (IN -- ---/OUT fl ---)
2013.03.06 11:03:35 MAPI XP Call: FlushQueues, hr = 0x00000000,
ulFlushFlags = 0x0000001a
2013.03.06 11:03:35 Sending one message
2013.03.06 11:03:35 Progress: Sending message 'Watch for this
message' (size 4.14 KBytes) Subject of the message so it is easy to
track
problem messages.
2013.03.06 11:03:35 MAPI Status: (IN -- ---/OUT fl act)
2013.03.06 11:03:36 MAPI Status: (IN -- ---/OUT fl ---)
2013.03.06 11:03:36 MAPI XP Call: SubmitMessage, hr = 0x00000000
SubmitMessage WORKED !!!
2013.03.06 11:03:36 MAPI XP Call: EndMessage, hr =
0x00000000
2013.03.06 11:03:36 FINISHED MAPI TASK
2013.03.06 11:03:36 Microsoft Exchange: ReportStatus:
RSF_COMPLETED, hr = 0x00000000
-
2013.03.06 11:03:36 Microsoft Exchange: Synch operation
completed
2013.03.06 11:03:36 Sending done, Error code = 0x00000000
2013.03.06 11:03:36 Sending done, Error code = 0x8004010f Normal
Error and Can be Ignored.
2013.03.06 11:03:36 MAPI Status: (IN -- ---/OUT -- ---)
2013.03.06 11:03:36 Finishing the Spooling Cycle, Error code =
0x00000000 Spooler Cycle Completed with No Errors
2013.03.06 11:04:10 Microsoft Exchange: Synch operation started
(flags = 00000001)
2013.03.06 11:04:10 Microsoft Exchange: UploadItems: 1 messages
to send
2013.03.06 11:04:10 EXECUTING Put MAPI TASK
2013.03.06 11:04:10 Starting the Spooling Cycle
2013.03.06 11:04:10 MAPI Status: (IN -- ---/OUT fl ---)
2013.03.06 11:04:10 MAPI XP Call: FlushQueues, hr = 0x00000000,
ulFlushFlags = 0x0000001a
2013.03.06 11:04:10 Sending one message
2013.03.06 11:04:10 Progress: Sending message 'This should
generate an NDR' (size 3.85 KBytes) Subject of the second
message
2013.03.06 11:04:10 MAPI Status: (IN -- ---/OUT fl act)
2013.03.06 11:04:10 MAPI Status: (IN -- ---/OUT fl ---)
2013.03.06 11:04:10 MAPI XP Call: SubmitMessage, hr = 0x80040502
Submit Message returned a REAL Error.
2013.03.06 11:04:10 MAPI XP Call: SubmitMessage returned
MAPI_E_NOT_ME. MAPI_NOT_ME means this transport cannot
deliver to this type of address.
2013.03.06 11:04:10 FINISHED MAPI TASK
2013.03.06 11:04:11 Microsoft Exchange: ReportStatus:
RSF_COMPLETED, hr = 0x00000000
2013.03.06 11:04:11 Microsoft Exchange: Synch operation
completed
2013.03.06 11:04:11 Sending done, Error code = 0x00000000
2013.03.06 11:04:11 Sending done, Error code = 0x8004010f Normal
expected error
2013.03.06 11:04:11 MAPI Status: (IN -- ---/OUT -- ---)
2013.03.06 11:04:11 Finishing the Spooling Cycle, Error code =
0x00000000 The Spooler Cycle finished successfully. Note
that an error sending one particular message
does NOT mean the spooler cycle failed. It
means the one message failed.
APPENDIX B SAMPLE SYNCHRONIZATION ISSUES FOLDER MESSAGES
OUTBOX, REGULAR SEND:
This is the log from a regular send that shows that the Outbox
is NOT part of the folder sync during a send:
12:27:44 Synchronizer Version 14.0.6133
12:27:44 Synchronizing Mailbox 'Test Account'
12:27:44 Synchronizing local changes in folder 'Outbox' Outbox
Folder, but no message to sync
12:27:44 Uploading to server 'server
sampleserver.sample.com'
12:27:44 Done
-
OUTBOX, ROGUE ITEM:
This is the log from creating a rogue message in the Outbox that
has nothing to do with submitting mail. I just
dragged an item into the Outbox folder and then deleted it.
Deletions from the Outbox folder are the one
corner case where sync does happen. Do not be confused by the
error message here. It is expected and normal
in modern Exchange scenarios.
12:30:49 Synchronizer Version 14.0.6133
12:30:49 Synchronizing Mailbox 'Test
Account'
12:30:49 Synchronizing local changes in folder
'Outbox'
12:30:49 Uploading to server 'server
sampleserver.sample.com'
12:30:49 Synchronization of some deletions
failed.
Normal error for this case. Really just a warning that
either
there was no message to delete server side, or the deletion
will
be done async and Exchange tells Outlook that it doesnt know
if it worked yet or not.
12:30:49 [0-130] This is just the error code for the above
error.
12:30:49 1 item(s) deleted in
online folder
12:30:49 Done
SENT ITEMS, REGULAR SEND:
This is an example of the successful sync of the Sent Items
during the sendone process. It would be great if
the logs showed this as a MOVE and not a full item upload, but
at least they show that the sync happened.
12:27:56 Synchronizer Version 14.0.6133
12:27:56 Synchronizing Mailbox 'Test Account'
12:27:56 Synchronizing local changes in folder 'Sent Items' Sent
Items folder
12:27:56 Uploading to server 'server
sampleserver.sample.com'
12:27:57 1 item(s) added to online folder One item syncd up
12:27:57 Downloading from server 'server
sampleserver.sample.com'
12:27:57 Done
-
APPENDIX C DIFFERENCES BETWEEN STORE AND TRANSPORT SEND
This table summarizes the differences between the Store Send and
Transport Send methods.
Topic Store Send Transport Send
Cached Mode Send method No Yes
Online Mode Send method Maybe, depending on other
options
Maybe, depending on other
options
Copying from Outbox to Sent
Items folder
Done by the Exchange Server Almost always done by the
Outlook
client
Send Immediately When
Connected
Not needed in basic Store Send Must be done to start a
spooling
cycle so the mail is sent now
instead of on the send/receive
schedule
Requires the spooler to run No Yes
NDR generation Done by the Exchange Server Done by Outlook OR
the Exchange
Server depending on the type of
NDR
Requires sync to work correctly
so that Sent Items work right
No Yes
Address Type importance No, Address Types are known and
used when needed by the server
Yes Valid address types must
have been retrieved from the
server during logon and be
available and present to Outlook
MAPI provider that does the
work
Emsmdb32.dll (The Exchange MAPI
provider)
Mspst32.dll (The OST/PST MAPI
Provider)
Send/Receive group settings Not important since no spooler
cycle is needed to send the mail
Could be controlling the send
schedule if you do not use the
Send Immediately when
Connected option