Enterprise reporting for mobility services · Enterprise reporting for mobility services involves considerably more than tallying the number of devices in use. In fact, a view into

Technical Deep Dive

Enterprise reporting for mobility services Has Microsoft EMS Intune grown up enough?Workplace & MobilityMarch 2018

2

The current Intune Data Warehouse does indeed contain a wealth of data, both in breadth and depth. The challenge lies in accessing and manipulating this data to meet the large-scale and detailed needs of enterprise mobility management and reporting.

Microsoft Azure portal and Graph API, designed for device operational management, can be used for reporting but are not fine-tuned for it and run out of steam when scaling up to enterprise requirements. Intune Data Warehouse and Power BI, designed for Intune reporting, are good tools with great datasets, but they require huge downloads to everyone who pulls a report and are therefore not ideal for enterprise reporting.

How, then, can an enterprise efficiently manage mobile devices? We have found that a custom reporting tool leveraging the Intune Data Warehouse and other data sources can overcome some of the inherent limitations of the existing tools and provide enterprise reporting, forecasting and analysis.

What enterprises need to know

Enterprise reporting for mobility services involves considerably more than tallying the number of devices in use. In fact, a view into a range of details is critical to enterprise and workplace security.

The ultimate goal of mobile device management (MDM) is safe, secure handling of corporate data by device users, and enterprise reporting for mobility should reflect that goal. Companies need to understand how the digital platform management services affect enterprise security and how they contribute to the safety, certainty and enjoyment of the mobile workplace experience.

When we purchase mobility management as a service rather than manage it ourselves, we want to know what we are paying for, how it is utilized and what quality of service we are receiving. We need comprehensive reports.

In an enterprise report, we are not looking at details of a specific device — those details support the day-to-day management and operations of the service; instead, we are looking for service health and service consumption indicators. We don’t want to know that “Joe Blog’s” iPhone was used yesterday at Walmart, but we do want to know whether the phone is compliant, secure and functional.

Ideally, we also want to get data from all devices that “touch” our infrastructure. The management service is a means to protect corporate data, and we want to know about any device that connects, not just the managed ones.

In the first releases of Intune — the component of Microsoft Enterprise Mobility + Security (EMS) that manages mobile devices and apps — it was clear that reporting was an afterthought, and reporting from the original Silverlight portal was difficult at best. But over time, Microsoft added more features — including the all-important device serial number — and populated the Intune Data Warehouse with more data than we ever could want. Does Intune finally provide enough data and facility to support enterprise reporting?

Technical Deep Dive

Managing large numbers of devices

In a small roll-out, we may know all of the mobile users and can anticipate their problems or risks. But when the number of managed devices grows, we lose visibility into the devices and the users. With a global rollout of thousands of devices, the small percentage of devices that are not compliant or have problems becomes a sizable number. You may need to know how many of these problem devices have been retired and how many have the latest patched operating system.

The report in Figure 1 shows 232 noncompliant devices, out of a total of 40,000-plus devices. When we had only 400 devices under management, there were only two noncompliant devices, and we could easily catch those two culprits when they walked into the office. But with 232 culprits, we need a different process. The process starts with defining what we need to know.

To effectively manage a large number of devices, we need reports on the number of available and assigned licenses for EMS and other mobility-related tools — and proof that the assigned licenses are really in use, because that’s what we are paying for. We need to know which devices are noncompliant and therefore pose a data risk. Some enterprise customers may even want to break these reports down by business unit or location.

Here’s what we need to know:

• Number of devices, both managed and not managed

• Number of devices, both compliant and noncompliant with corporate requirements

• Breakdown by owner type, including bring-your-own (BYO), corporate personal enabled, corporate shared and corporate kiosk

• Age of the corporate devices in use and when they will need to be replaced

3

Figure 1. Sample report for mobility service consumption

Technical Deep Dive

MES Usage Report at: Feb-18 Jan-18 Dec-17 Nov-17 Oct-17 Sep-17 Aug-17 Jul-17 Jun-17

Total number of users: 40,232 38,791 38,486 36,789 35,409 38,486 36,789 35,409 34,078

# Users never connected 537 415 452 330 525 452 330 525 720

# Users with pending activations 232 342 354 232 379 354 232 379 562

# Devices that are wiped/disconnected 232 208 208 61 391 208 61 391 427

# Other devices not active in last 90 days 305 110 305 49 305 305 49 305 415

# New or reactivated devices in last 30 days 3,736 3,407 3,626 3,895 4,811 3,626 3,895 4,811 4,554

# Non-compliant devices 232 232 220 208 208 229 216 209 202

Users licensed but never connected (after 90 days):

[email protected]

[email protected]

…

[email protected]

• Changes over the last month, including new devices, new enrollments, removed devices and lost/stolen devices

• Number of incidents logged with the service desk regarding mobile devices and the cost of repairs and device downtime

With all this data, you need some guidance regarding what is acceptable. It’s helpful to know, for instance, if the number of noncompliant devices is improving and how the service last month compares to this month’s service. Figure 1 gives a sample report with some key service parameters and compares the latest numbers — in this case, February 2018 — with those of previous months.

How to use the data efficiently

For the reporting service, we do not want to add a data collection agent to the devices, but rather, reuse data collected by existing services. When a device is used, it talks with the back office, Active Directory, the device management service EMS and, most importantly, with the Microsoft Exchange server (Figure 2). Every interaction leaves fingerprints in log files, and we can use these log files for reports so that the reporting service has zero impact on the user. We want to use all available data to define the service health. If we want to know whether a device is still in use, for instance, the back-office logs may be more reliable than the EMS logs.

We also have security matters to consider. At a minimum, we want all data encrypted both in situ and in transit. We might also need to obfuscate personally identifiable information (PII) data and make sure we can easily remove PII historical data so that we can be compliant with General Data Protection Regulations (GDPR). We also may want to have access to historical data so we can compare today’s service with last month’s or last year’s. We may need to know whether the process scales securely to a large number of devices.

Figure 2. Data flow from collection by a managed device in EMS until reporting

4

Technical Deep Dive

EMS

Back o�ce

Portal

Data warehouse

Email

Logs

Logs

Logs

Logs

Managed device

AAD

If we look at the process of creating the report, we need to know whether the system:

• Runs reports fully automated, with no need for user interaction; and enables “set and forget” report creation, so that when an administrator is on extended or sick leave, a report is still created

• Accommodates both ad hoc reports and reports scheduled for a specific time — the first Monday of the month, for example

• Requires administrative privileges to run the report or accepts a normal or reporting service account — and whether it exposes the passwords for these accounts

And finally, we want the report to be repeatable. If two people create a report, the results should be the same. Ideally, we want to give managers the right to look at the service performance whenever they wish. They should not see data related to the day-to day-service operations, but instead should receive a report with an appropriate management summary — a clear snapshot of service health.

Azure portal and Graph API

So how well do Azure portal and Graph API meet these enterprise reporting requirements? As we mentioned earlier, they are designed for operations, where someone is looking at details one device at a time and performing actions based on that information. It’s not surprising, then, that the tools have trouble scaling for enterprise device management reporting.

That said, Azure portal is the second iteration of a management portal for Intune and a huge improvement over the Silverlight-based portal. You can export the “All devices” data view to Excel, as shown in Figure 3. The option is labeled “Export all,” and that’s exactly what it does. Azure portal exports all data — 36 fields, including serial number and email — from all devices. In the portal, you can customize the fields displayed and filter the rows, but the report exports all of the fields. That’s a problem when you have over 25,000 devices.

5

Technical Deep Dive

Figure 3. Intune reports in Azure portal

The exported data provides a good base for an enterprise report, but because its purpose is operational management rather than enterprise reporting, it cannot be scheduled. It can only be created ad hoc by someone with administrative privileges in the Intune Azure portal. In addition, the Azure portal contains “live” data, so if we create the same report 5 minutes apart, the details in the report will be different. Some devices, for instance, will have a “last contact date” in the 5 minutes that have lapsed since the initial report.

The move to the new Azure portal means that Graph API becomes available to access Intune data, and Microsoft supplemented Graph API with a set of PowerShell sample scripts. Graph API accesses the same live data as Azure portal and actually makes up many of the portal’s calls. The primary focus of Graph API is therefore the same as that of the Azure portal — operations.

A great advantage of Graph API is that the user requesting the data through a report doesn’t need access to the Azure portal. We can leverage Azure Automation — a cloud-based Azure service used to execute PowerShell workbooks “server-less” on a schedule — and use PowerShell to access Graph API. This process helps us collect the data at the same time as the previous collection, resulting in consistency. The collected data is copied into Azure SQL Database or optionally, moved into Azure Blob storage. We store variables and passwords in Azure Automation, encrypted if required, so the same script can be used for multiple environments. We can use a limited administrative account especially created for this task, so its password does not need to be stored anywhere else.

One of the biggest issues with Graph API is that the required reporting fields are available only in the beta implementation, which is, by its nature, a moving target. What works today might work differently tomorrow. Currently, most of the data in the Azure portal under “devices > All Devices” is available by querying https://graph.microsoft.com/beta/deviceManagement/manageddevices. Other queries are also available, mainly in relation to user, application, device and other policies. For several resources, Microsoft prevents overuse by adding throttling, including for listing device users from Azure Active Directory (AAD); but Intune device data is not controlled in this way.

Time-consuming data collection

Another big issue with Graph API is the number of calls required to obtain information for reporting.

To query Intune with PowerShell, I used early Microsoft examples in ManagedDevices_Hardware_Get, and modified the authentication to be silent/unattended, suitable for Azure Automation. I added paging, Azure Blob support and Azure SQL Database support to the PowerShell script. The first step is to collect a list of devices with their information; with a maximum of 999 devices per call, we require 11 calls to collect 10,000 devices. In the early Microsoft examples, we needed to query “advanced” details per device to get the serial number, user principal name (UPN), etc. The ManagedDevices_Get example, for instance, makes a per-device call to get the UPN. Most of these advanced details are now retrieved in the one call, but it is still a mystery why the device OS Edition and OS Language require a per-device request. In general, I found that the data in the subcollection “hardwareInformation” is reliable only if you ask for it “per device,” which would require a lot of API calls.

6

Technical Deep Dive

Apart from this, we haven’t collected all the data yet, we haven’t looked at EMS license assignments and we have no data to tell us which users have assigned licenses but aren’t using them.

So, clearly, Graph API has issues of scale that make it nearly unusable for enterprise reporting. Of course, it’s not fair to look at these as limitations of Graph API, since we are using Graph API here for something it was not intended to be used for.

Microsoft’s reporting tools for EMS Intune

While the focus of the Microsoft Intune Service Dashboard and the underpinning Graph API is to manage devices, the purpose of the Intune Data Warehouse, combined with Power BI, is to provide reports.

The Intune Data Warehouse consists of many linked tables, providing a rich dataset both in breadth (number of fields/attributes it can report on) and in depth (historical data). The abundance of data fuels powerful custom reporting.

The Intune Data Warehouse, however, is not a reporting engine. It contains the data that makes up the report, but you cannot manipulate the data in the warehouse. You need to copy the data over to work with it, since the Data Warehouse actually contains a copy of the Intune data and is therefore “read only.” Furthermore, the copy is not continuous, but rather, a once-a-day snapshot. It basically contains yesterday’s report — and a report for every day of the 90 days before that — in great detail.

Figure 4. Data schema from the Intune Data Warehouse

7

Technical Deep Dive

Accessing the Data Warehouse

The easiest way to access the Intune Data Warehouse is by downloading the Power BI file from Azure portal and running it with the Power BI Desktop. Each person who needs reports must use Power BI Desktop with a report template to create their own report and customize it. This means that each person must connect to the Data Warehouse and download their device’s own dataset, which is a big disadvantage for two reasons: (1) We are not looking at one consistent source, and (2) there is a hefty quantity of data to be downloaded — for 10,000 devices, the dataset is several gigabytes in size. Plus, this download happens only when the user clicks the refresh button, which is not ideal — although it might be a blessing, since the action triggers such a large download.

After downloading and connecting to the Intune Data Warehouse, we immediately see the prepackaged view of the device management data (Figure 4).

The Power BI Report template consists of a set of tabs/pages, and each tab has visualizations. Figure 5 shows visualizations for “Manufacturers,” “Device Ownership,” “AAD Compliance” and “Devices.” Once a visualization contains the data required for the report that you want to share with management, it is easy to export it to Excel.

Although the dataset in the Intune Data Warehouse is great, it is still limited to the underlying sources of Intune and AAD, with a bit of Office 365. You still need to combine it with other sources, such as procurement for device age and warranty information, or Service Now for incident and repair data. These additional data sources provide critical insight into the service health of the mobility service.

In summary, Power BI is a great tool to visualize service data, and it provides a useful pre-canned report for management; but it is not yet ready to meet the requirements of enterprise reporting.

Figure 5. Power BI presentation of Intune data

8

Technical Deep Dive

Extract Secure Publish

Staging BI4M Core ReportingData Sources• Intune• Office 365• AAD• OMS

Extract and transform data• Azure Data Factory• Azure Automation• SQL procedures, functions and views

Store in DB vault• GDPR complient• Multi-tenant

Analyze• Business insights

Publish reports• Service consumption• Service performance

Segregate• By customer• Department

ConsumeReports using• Power BI• Excel

Infrastructure: Azure SQL Databases

Figure 6. Logical view of device data flow from source to report

Create a custom reporting tool with PowerShell

Microsoft suggests that you can connect the Intune Data Warehouse to Power BI, Excel or any another analytics tool that supports OData feeds, so we tried to use the built-in OData connector in Azure Data Factory (ADF). That didn’t work. However, PowerShell turned out to be a good option.

Just as with Graph API, we can launch the PowerShell script from Azure Automation and either push the output to Azure Blob storage after compressing it or move it to an Azure SQL Database. The great advantage of using PowerShell is that we have full control of what data we want to use. We don’t need to upload 90 days of historical data every day, for instance — just the delta, or one day’s worth. Data that hasn’t changed typically doesn’t need to move again.

To set up reporting, we created our own data warehouse with targeted service reporting, where users can access pre-canned reports and the underlying data when they want, with the tools they want, including Power BI. This data warehouse takes in more than Intune data. Ideal candidates are procurement data, incident and repair data, and data collected by Operations Management Suite (OMS). In the design of this data warehouse, we anticipated different security roles, which we translated to different Azure SQL Databases on different servers. Since the databases are very lazy — that is, hardly any IO throughput is required — this is a very cheap but very secure solution.

We separated the staging database and the extract, transform and load (ETL) processes from the core database, a.k.a. the Vault, and created yet another database that contains reports ready for consumption. The reporting database can be fronted by Power BI or Excel for canned reporting.

We use this custom DXC Technology Mobility Data Warehouse for “standard” reporting in our modern management service. It is also leveraged by DXC Business Insights for Mobility Service (BI4M), where we baseline the service and add predictive analytics to help in forecasting, service health and performance analysis. This can be especially helpful if we add incident data and application usage data.

This solution finally meets enterprise reporting requirements. It builds on top of the reporting engine from Microsoft but adds more sources to provide a complete overview of service health. Users don’t need to download all gigabytes of data, just their favorite service health report. If users expect only last month’s service report,

9

that is the report they receive. If they are entitled to yesterday’s service health report, that is the report they can download. This solution is both standard — meaning the same set of reports is created for every customer — and custom: We can adapt to the customer’s reporting requirements. The data vault is open to inject more corporate data if the customer wishes, or the customer’s data analysts could use existing reporting data for further research.

Conclusion

So did EMS Intune finally grow up? Is it capable of meeting the comprehensive and large-scale needs of an enterprise? The answer of course is: It depends. If you use Intune as a tool to manage and collect an inventory of your enterprise devices, then yes, Intune has grown up — and is still growing. However, real enterprise answers require data that sits outside Intune, even outside Azure. Intune can definitely contribute its bit, but it does not provide the complete answer. For a complete solution, you need a holistic view such as the one provided by the custom DXC Mobility Data Warehouse.

About the author

Ben Santing, a technologist/senior technical architect for Workplace and Mobility at DXC Technology, focuses on transforming cutting-edge technology into services that provide business-critical functions for enterprise customers. His interest in the bigger picture is matched by technical skills that include Windows NT MCSE, ITIL V3 expertise, IT Strategy and Architecture certification and Azure EMS MCP. His skills are demonstrated in various knowledge briefs and white papers. He held a variety of architecture and engineering positions in DXC Workplace and Mobility before becoming a lead architect for DXC Device as a Service and DXC Business Insight for Mobility. Ben lives in the Republic of Ireland and originates from the Netherlands.

Learn more at www.dxc.technology/workplace_and_mobility

www.dxc.technology

Technical Deep Dive

How we created a custom DXC Mobility Data Warehouse

To create the custom data warehouse solution, we built on Microsoft’s “IntuneDataWarehouseCmdlets” examples, borrowing from previous work with Graph API. We added silent authentication, Azure Blob support with file compress and Azure SQL Database support with automatic table creation to the PowerShell script. We also added a text cleanup and text trimmer to make copying to the Azure SQL Database easier. Although the example comes with paging support, we did not find a maximum to the number of rows that can be returned. If there is, it would be well over 10,000.

If you want to get everything out of the data warehouse, it will take longer than an hour, which is the time limit on your authentication; so, we changed the script to reauthenticate before each new collection.

From the Data Warehouse, the “Device” table or the “mdmDeviceInventoryHistories” table is of most interest to us. The tables labeled “histories” contain 90 days of histories. The first column is “datekey” (e.g., 20180114), so it is easy to filter on data that we haven’t collected before. There are other tables, such as the “‘userDeviceAssociations” table or the “currentuser” table, that should be used for final reporting.

During the ETL process on the staging database, we add our own tags to each table row — customer ID, source ID, report date and time at source, and row insertion date time. We move any PII reference, including the device name, to PII labeled tables, as most of the management reports are aggregated reports and don’t require any PII data. This makes it easy to obfuscate PII data and to archive PII data older than 3 months, in compliance with GDPR. While we combine most of the device data from different sources and customers, PII data stays segregated by customer. On the staging database and reporting database, we separate data using different SQL security schemas for each customer.

Locations and addresses can be PII data, but we still want to have some geolocation information available for each device. The simplest method is to create location keys based on the address area’s ZIP code. For the DXC Technology headquarters in Virginia in the United States, for instance, this would become USVA22102. For the office from which I work in Ireland, it would be IEH91.

ADF triggers automated data transformation, which uses SQL table views, procedures and functions. ADF is also responsible for moving the data from the ETL database to the Vault and from there to the reporting database. We make use of Azure SQL security features such as F to protect the Azure SQL Database.

© 2018 DXC Technology Company. All rights reserved. MD_7963a-18. March 2018