BACKGROUND AND PURPOSE
One of the top challenges for Enterprise customers in their journey to Office 365 and Cloud based services is the transformation needed to handle the constant service Changes that make out a natural part of SaaS and the Cloud. The concept of “evergreen services” is still for many completely new, and customers often finds this parameter more challenging than the technical implementation and onboarding itself. Microsoft recognizes this, and are committed to help organizations prepare for change by implementing various change control mechanisms to the service. They are also more transparent to their customers than they have ever been in regards to Product roadmaps (see the Office 365 Public Roadmap here) and Community presence (see the Office 365 Network on Yammer here). Regardless of all these efforts from Microsoft, Enterprise customers that are new to Office 365 always asks me and my team for advice in this complex topic. The question is initially almost always formulated in the same way: “How do we create the test Environment for this?”. The answer: “It’s not that simple…”, and then the discussion follows, uncovering all aspects from handling operational maintenance to planning for unavoidable updates in the user interface.
The purpose of this post is to give an overview to how Enterprise customers should comprehend and plan for ongoing change after Office 365 is implemented and all the users are onboarded to the service. Microsoft provides good information in their service descriptions, on the Office Blogs etc, but this post is to be considered as “the lost documentation” of the holistic overview that often is requested by customers when they are new to Office 365.
OFFICE 365 IS “EVERGREEN” – WHAT COULD POSSIBLY GO WRONG?
The concept of Evergreen means that the service will constantly improve and evolve, without you as a customer having to worry about it. The reality of it is that you do not need to take care of server version upgrades, security patches, feature implementation or maintenance – your service provider takes care of that for you. This will save you time and money, not least as you never have to run an upgrade project again for i.e Exchange Server or Skype for Business Server etc. On the other hand, if your organization is not used to rapid releases and always being on the latest version, you will need to shift your mindset from “big change projects every five years” to “smaller updates every month”. The concept is often referred to as moving from “waves to ripples”, a good metaphor to how things will change after the move to Office 365.
Evergreen is great, however it will put new requirements on your IT-organization if you are not used to deliver changes rapidly. This scares many Enterprise customers, and sadly some are disqualifying Office 365 just because they think that they can’t control change in Office 365, and that “a high release pace determined by somebody else” is therefore nothing for them. That is a false apprehension, as there are many ways to control change in Office 365, which will be covered in this post.
As the World is changing and millennials are entering the workplace, the pressure from the business on the IT-organization to deliver a modern digital workplace is often very high. Also, providing modern productivity- and collaboration tools is often key to prevent rogue IT, where users are adopting Commercial solutions for cloud storage, or send sensitive information to Commercial e-mail services so that they can access it anywhere on any device etc. These business requirements goes hand in hand with what Office 365 offers, so the IT-organization will need to deliver, and at the same time manage change in an Evergreen service. Let’s have a look how this can be accomplished in practice.
BECOME AN “EVERGREEN READY ENTERPRISE” WITH OFFICE 365 – CATEGORIES OF CHANGE
I will explain how to manage change in Office 365 by breaking the topic down to categories. I will explain the challenges and remediation options for each category. If you have ideas for other categories, please reach to me on Twitter (@JesperStahle), and I will consider adding them to this post. My intention is to keep the documentation alive and updated as Office 365 is changing.
CATEGORY #1: THE ROLE AND RESPONSIBILITY OF THE SERVICE OWNER
CHALLENGE: The shift to become “Evergreen ready” starts with the role of the Service Owner (sometimes referred to as Service Responsible or Service Manager, depending on how ITIL-influensed operations model you might have). The person owning the Office 365 service, regardless if it is the whole suite, or independent services (such as Exchange Online, SharePoint Online, etc.), must be prepared to become proactive in a whole new way than before. Traditionally, the Service Owners might find themselves in a situation where there are several months or even years between service changes, and the role is often described as more reactive than proactive. When a change is needed due to new dependencies or business requirements, the Service Owner would be notified and take the necessary decisions and issue transformation projects.
When the Service Owner becomes responsible to deliver an Evergreen service within agreed SLAs, it will be required that he or she becomes more pro-active to be notified of upcoming changes, before they are implemented and affects end users and the IT-operation. If the notifications are missed or ignored, the Service Owner will one day have a change issued outside of their control, which might cause frustration from both IT-operations and end users, and in the worst case even impact the SLA for which the Service Owner is responsible for.
BECOMING PROACTIVE ENOUGH: It is vital that the Service Owner of Office 365 suite, and/or its components, becomes proactive by monitoring all the communications from Microsoft regarding upcoming changes. Microsoft will not implement changes in Office 365 without first notifying their customers, but if you do not have a proper ownership and process for reading and responding to these communications they are to no use for your organization. It is a bad practice to trust the administrators of Office 365 to monitor the communications about upcoming changes and react to them accordingly – the administrators should put their focus on administering the service and enabling new or changed features as that is ordered/issued by the Service Manager. A better practice is to have a named Service Owner that takes responsibility for service communications from Microsoft as well as monitoring the Office 365 Roadmap to plan for upcoming changes and feature introductions.
In practice, this means that the Service Owner should monitor four main communication channels and implement processes to be able to rapidly respond to the changes announced in them. Here is the list of the four main communication channels:
- The Office 365 Message Center: In the Office 365 administration portal there is a Message Center that is used to notify customers about upcoming change and incidents. The Service Manager can access the Message Center by acquiring an administrative role in the Office 365 tenant (i.e the Service Admin role) then visit the Office 365 Admin Portal and navigate to Message Center. Here is a direct link. Here is an example screenshot of how the Message Center looks like (click picture to enlarge):As you can see, there are different categories of information. Not every communication in the message center is necessarily targeted for the Service Owner or similar function. Here is an overview of the Message Center taxonomy (click picture to enlarge):It is up to every organization to shape their own RACI matrix for these communications, but it is generally recommended for the Service Owner to be responsible for communications classified as “Stay informed” and “Plan for change”, and to be informed about communications classified as “Prevent/Fix issue” and “Service Incidents”.
- The Office 365 Public Roadmap: The next communications channel the Service Owner should be responsible to monitor is the Office 365 Roadmap, found here. This is a great evidence that Microsoft is aware that change requires planning. The roadmap site let customers see which features are in development and which that are currently being rolled out. When features are due to be rolled out, you will also be notified in the Message Center, so the main reason why the Service Owner should have the ownership and responsibility to act on this communication channel is for the organization to be aware about features that are in the “In development” category. By monitoring this category, the Service Owner can avoid surprises in the Message Center, and will have much more time to plan for change. Here is an example screenshot of how the Office 365 Public Roadmap looks like, note the category “In development” (click picture to enlarge).
- The Office Blogs: The third communication channel every Office 365 Service Owner should monitor is the Office Blogs – found here. Here you will find various communications from the teams at Microsoft that build Office 365, not just feature announcements, but also interesting costumer stories, news round-up articles and video casts (Office Mechanics) covering different a wide variety of topics. A good advice is for Office 365 Service Owners to have the Office Blogs as their start page in their web browser.
- The Office 365 Network on Yammer: With over 89 000 members and growing, the Office 365 Network is a community for Office 365 customers and IT-professionals world wide. Service Owners can participate in various groups within the network to discuss Office 365 topics. Microsoft themselves has a strong presence here, and often customers find their questions answered directly by representatives from the Microsoft product groups. This is a great forum for Service Owners as they can learn how others are solving issues that they are facing. The recommendation for the Service Owner is to monitor the network (and/or it’s quarterly digests known as the Office 365 Network Bulletins), but it is not necessary to read all its content daily. The Office 365 Network is found here.
CATEGORY #2: PROCESSES FOR SERVICE UPDATES, FEATURE INTRODUCTIONS AND UI CHANGES
Release Options: In Office 365, updates and new features are released constantly. Staying informed through the communication channels helps you prepare for change, but in order to be fully prepared processes are required for testing and evaluating new functionality. Therefore, it is possible to nominate users in the organization to be part of the First release ring (wave) for updates. First release users will have new features and updates before other users, so that they can test the updates in production and prepare integrations and documentation, such as end-user communications.
It is also possible to nominate all users in the tenant to be part of the first release ring by enabling First release for scope of the Entire organization. That would give everyone access to new features sooner, but will give the Service Owner and the IT-organization less time to prepare for change, which is not optimal for Enterprise customers. In May 2015, around 10% of all Office 365 customers had enabled First Release for the Entire organization, not just for selected people.
Find more information about the First Release options here. Here is an example screenshot of the Office 365 release options and how they are configured in the Office 365 admin portal (click picture to enlarge).
End-user awareness: Upcoming change is announced through the communication channels, and will be available for test and evaluation for the users nominated to be part of the First release ring. Those individuals should in concert with the Service Owner evaluate all dependencies to each new feature and update, as well as assess if end-user communications will be necessary. Where end-user communication is necessary, Microsoft often provide templates and input for your organization, so you only need to review the material and make your finishing touches to it.
The Skype for Business example: A good example of a change process was when Microsoft changed the name and user interface to Skype for Business from what was formerly known as Lync. Microsoft first announced this in the Roadmap site and on the Office Blogs, then shortly after in the Message Center under the category “Prepare for Change”. All the information needed was provided for the Service Owner to plan for change, and a preview version of Skype for Business was made available to install for the First Release users (or anyone else who wanted to try it). Microsoft prepared end-user information in both written form and in video format (see picture below) for customers, ready to be distributed directly to end users. The article with that material can be found here for reference. If any customer had any questions about the update – before, during or after – they could ask them in the Office 365 Network on Yammer and have the discussion going, and a potential answer, moments later.
CATEGORY #3: PROCESSES FOR OFFICE 365 PROPLUS UPDATES ON CLIENTS
Whatever you did before will work: A common misunderstanding is that if you deploy Office 365 ProPlus (Click-to-Run), you have no control of updates and your users will have features introduced without your control. The reality of it is that any change/update process you had for MSI-based distribution of Office can be used with Office 365 ProPlus Click-to-Run, you just have to adjust your distribution method a bit. You can distribute Office 365 ProPlus using SCCM (good information in this article) and you can then distribute updates in a staged fashion with test rings/waves, just as you probably did before. If you have SCCM Version 1606 or later, you can distribute the updates using SCCM (see details below).
Control distribution of updates without SCCM: When deploying Office 365 ProPlus, you can use a configuration file (config.xml), or Group Policy settings, to configure the location for updates for individual computers using the UpdatePath parameter. Read more about the config.xml parameters here, and the GPO settings here. The UpdatePath will point to a file share on the network where binaries for the latest available build of Office 365 ProPlus will be located. It is up to every organization to plan and design the file share – you can use shared folders on a DFS file share or individual file shares distributed on site-local file servers etc. You can have separate file shares for pilot groups that can evaluate the Office updates (you place new updates here first), and these updates can then be moved to the production file share that are used by the majority of the users. This creates an environment where you can stage the updates and evaluate them before distributing to the masses, which also facilitates end-user communications if needed for user interface changes etc. To generate the config.xml, I recommend using this XML Editor tool.
Control distribution of updates with SCCM: If you have SCCM, you can use it to distribute updates. It is recommended to use the methods explained here on SCCM 1606 or later, but if you are on version 1511, you can use functionality that will remain in preview explained here.
Updates can be postponed, but to stay fully supported you should deploy Office 365 ProPlus updates within 12 months from when they were released. Security updates and feature updates are today bundled together, but options for separating them will be available soon. Regardless of that, the recommendation is to evaluate updated versions in the pilot ring immediately when they are available, and distribute them to the production ring as soon as possible after the verification. Consuming cloud services requires the client to be up to date to have full support, optimal performance and all features available – try not to fall into old habits and postpone updates just because you have the option to do so for 12 months.
First Release users (see Category #2) will have the option to download Preview versions of Office 365 ProPlus to test functionality that is due to be released in a bit more distant future. The First Release users can sign in to the Office 365 Portal and download the Preview version from the Software section (direct link here) – just scroll down on the page, the Preview version is located under the download section for the current version. The Preview versions will never be displayed for users that are not in the First Release ring.
An example overview of the non-SCCM solution (Group Policy based): In the graphics you can see a file server hosting two file shares containing different versions of Office 365 ProPlus – one for the Pilot Ring and one for the Production Ring. The binaries (bits) are downloaded from Office 365 on the server (can be automated with scheduled task). Users in the Pilot Ring have their UpdatePath parameter set to the Pilot-share, and the users in the Production Ring have their UpdatePath parameter set to the Prod-share. In the First Release Ring, users are downloading Preview bits from the Office 365 Portal under the Software section. Click graphics to enlarge. Note: If you have SCCM, the native methods referred to above (“Control distribution of updates with SCCM”) is recommended.
CATEGORY #4: MANAGING CHANGES TO YOUR ON-PREMISES FOOTPRINT (ADFS, AADSYNC, EXCHANGE HYBRID, ETC.)
Enterprise Office 365 deployments are integrated with the on-premises infrastructure through an “identity bridge” (synchronization and federation, also known as AADSync and ADFS) and sometimes an Exchange Hybrid configuration. There can also be integrations with Lync/Skype for Business (Hybrid Configuration) as well as Search integrations between SharePoint Server on-premises and SharePoint Online.
Test environments for this can be provisioned, but it does not makes sense to do so if the on-premises system in question does not have a full test environment to begin with. As an example, if you don’t have a test environment for Active Directory before introducing the identity bridge to Office 365/Azure AD, it does not make sense to provision a test environment for the AADSync and ADFS components, as they would read and write to the exact same source and target as the production instance. If, however, you do have a full featured test environment for Active Directory, you could of course choose to provision a separate Azure AD tenant to which you integrate the test environment through an identity bridge in the test environment.
The same logic goes for Exchange, Lync/Skype for Business and SharePoint – if you don’t have proper test environments to begin with, it would not be possible to produce a fair test environment for the Office 365 integration.
If the intention is to create a test environment just for AAD Connect and the Office 365 tenant, Microsoft have published step-by-step instructions on how to achieve that in a Microsoft Azure environment here.
Different support statements are true for the on-premises components. Here are the main guidelines to follow:
- AADSync / DirSync / AADConnect: The installation of the sync tool is meant to be “appliance like”. You do not necessarily need to update to every version that is released to stay supported unless you’re noticed to do so (all though it is generally recommended to stay up to date from the feature- and performance perspective). You can choose not to upgrade the sync tool until you are implementing a certain feature that requires a certain version – or until you are receive a message in the Message Center (see Category #1) saying that you need to upgrade to stay supported. The general recommendation is for the Active Directory Sync Tool Service Owner to monitor the Version Release History article, found here, to be aware about when updates are released and plan for installing the update accordingly. If test environments are not used, there are is a “safety net” measure that customers can take to prevent configuration errors in the sync tool to soft delete users in Azure AD. It is called PreventAccidentalDeletes, more information can be found here. As mentioned above, a test environment for AAD Connect and Office 365 tenant can be built in Microsoft Azure following the steps outlined here and here.
- ADFS: Like with Active Directory and Windows Server in general, it is recommended to keep ADFS updated through Windows Update. This is recommended for security reasons and to ensure optimal performance and functionality. So far, all versions of ADFS that have been released since Windows Server 2008 (ADFS 2.0) have been supported in Office 365, so you may consider your installation fully supported as long as you are installing all Windows Server updates from Windows Update, and as long as the version of Windows Server that you are running is in Mainstream Support (find out more here). However, new ADFS-features are available with new versions of Windows Server, which may encourage you to upgrade your ADFS environment as well. An example of such feature that have made many ADFS 2.0 (Windows Server 2008/2008R2) customers upgrade to ADFS 3.0 (Windows Server 2012R2) is the Extranet Lockout feature (read more here).
- Exchange Hybrid Configuration: To stay supported in an Exchange Hybrid Configuration, your on-premises Exchange Server environment must be on a supported version (at the time writing Exchange 2007 SP3RU10, 2010 SP3 and 2013 CU7, where 2010 and 2013 can be the Hybrid servers facing Exchange Online) and the latest available build minus one cumulative update should be installed – also referred to as the “N to N-1 guideline”). This means that if you have an Exchange Server 2010 SP3 environment on-premises, and the latest available version is Exchange Server 2010 SP3 Update Rollup 10, you would have to have at least Exchange Server 2010 SP3 Update Rollup 9 to be fully supported. The written support statement can be found here, and it actually does not mention the N to N-1 guideline. However, if you run in to issues and attempt to open a support ticket, Microsoft will remind you of the N to N-1 guideline and encourage you to update to the N to N-1 build and reproduce the issue on that version before troubleshooting continues.
- Lync, Skype for Business and SharePoint Hybrid Configurations: For these workloads, the support statements are not as clear as with Exchange etc. The general recommendation is to keep the on-premises server products on the latest available build (install available updates) and monitor the Message Center for notifications about supportability.
CATEGORY #5: DEVELOPMENT, CUSTOMIZATIONS AND BRANDING
There is one category where test tenants makes absolute sense – and that is for system development. In the perspective of Office 365 as a platform for developers, providing a test tenant is absolutely necessary. Everything from branding the suite bar to releasing code in SharePoint Online benefits from being tested first. However, these test tenants do not require a full identity bridge etc to do their job – instead developers can provision their own test tenants from their MSDN Subscription benefits (or even Trial plans), or you can provision one on a separate subscription. This test tenant can be completely out of scope for the Service Owners responsibilities. As mentioned for AAD Connect above, another option for creating a development environment is to leverage Microsoft Azure by following the steps outlined here.
SUMMARY AND DISCLAIMER
There are many misunderstanding when it comes to Office 365 and managing change in the Enterprise. Hopefully this article can help som organizations realizing that the Cloud, SaaS-apps and “Evergreen services” are manageable even in an Enterprise environment, and that good instruments exist so that the Service Owner can master even a very rapid release pace. Please note that the information given in this article is to be considered as guidelines shaped from real-world experience – but that change management is a complex topic that always have organization specific requirements. Modernizing change management to be “Evergreen ready” is not always easy, and it will most probably not happen over one night – but it is absolutely possible. Hopefully content of this article has helped proven that for you.
And remember: change is good!