Wednesday, September 23, 2015

Sitecore Deployments using OctopusDeploy - Part 2 (Deployment Process and Considerations using OctopusDeploy)


Part 1 | Part 2 | Part 3 | Part 4

In my first post I had mentioned about typical Continuous Integration and Deployment architecture, considerations for development and Continuous Integration. In this 4 part series, this is the second post and I am talking about Deployment process and various considerations that needs to be made when using OctopusDeploy


OVERVIEW


OctopusDeploy is a nice deployment frameworks which works against Nuget packages. If you are not already using any deployment automation platform then you should definitely consider OctopusDeploy as a platform, not only for Sitecore but also for enterprise wide Microsoft solutions delivery.

OctoupsDeploy has two components. One is OctopusDeploy server and other is group of tentacles (agents running on your instances). The communication is secure by tentacle authenticating the server over X.509 certificate. More on this here



I am listing down few features which would be useful in your Sitecore automated deployments using OctopusDeploy


  • Configuration for high availability. Find out more about this here
  • Rolling deployments - Useful when you are doing web package deployments across CM or CD nodes in a clustered instance. More information about this here 
  • Ability to execute PowerShell commands on the Tentacles
  • In-built Nuget feed which would be enough for small engagements.
  • Deployment dashboards, showing which release is deployed in which environment
  • Ability to configure multiple tentacles on the same server, tag different machines with different roles
  • Ability to do configuration transformations at the time of deployment to specific environment.
  • Ability to listen to any external Nuget feed.
  • Ability to create variables and scope it at each Environment, Role or Machine level or combination of it, to define contextual values. This comes very handy to make your deployment process generic and repeatable.
  • Rich set of script templates allows you to build the deployment process pretty quickly. You can check this library here
  • Supports Windows Azure deployments
  • Ability to create automated releases, which will be helpful if you intend to setup continuous deployment to your pre-production (development / demo) servers.
  • Integration with your existing Active Directory. 
  • In-built roles allows you to restrict access to projects, environment configurations and others. Read more about security in OctopusDeploy here


PRE-CONSIDERATIONS


Before or during the process of deployment automation, consider below mentioned aspects 


  • If you are working in multi-vendor kind of engagement wherein infrastructure is owned and maintained by some other agency, its better to get them involved early in the process
  • You will need few service accounts with specific permissions to folders on each VM, execute PowerShell to deal with IIS, Nuget package deployment, restart app pools, read certificate store etc. 
  • You will also need a Sitecore user who has access to install packages, issue publishing and indexing requests as part of deployment process.
  • You need to open few inbound ports on each VM so that Octopus can communicate with its tentacles for deployments and health checks
  • If you also have steps for database backup / script executions / restorations, consider a SQL Server user having that level of access.
  • Try and script out your tentacle installation. Depending on number of machines and environments this can be error prone if you do this manually. You can get some help here
  • Store your OctopusDeploy master key safely. This will be handy if at all you need to restore the old db backup. Note that Octopus deployment database is encrypted. Without Master key, you will not be able to restore old data
  • Create test Sitecore deployment servers to ensure your deployment process works once its ready and also during the instrumentation phase so that you have some place to do try things out.
  • Plan for testing all the tentacles in all your environments. Even though you tested your deployment process in test environment, you do not want to try deployments first time in production without ensuring if the tentacles and all necessary permissions are granted as expected in those machines in Production.
  • Find out size of your packages and how long it takes to deploy / install your packages. You may need to tweak some timeouts at firewall or app pool level to be able to handle medium / large packages if those take beyond 5 minutes or so.
  • Give due considerations on how would you deploy web packages (not Sitecore). Do you have distributed file sync mechanism? or do you want to rely on Octopus? We got away with file sync mechanism and handled all package deployments using Octopus. 
  • If so, you also need to think about how would you deploy Sitecore Modules or hotfixes received from Sitecore? They usually many not be available in Nuget package format. We addressed this by developing our own tool to convert and package anything on the file system to Nuget package.
  • Consider all your deployment scenarios. In our case we had so many
    • Core Platform packages
    • Brand platform packages 
    • Hotfix packages
    • Sitecore Modules, Patches / Hotfixes in different formats
    • Small Configuration changes (such as tuning Sitecore configs, changing log levels)
    • Site specific configuration changes

I think that's quite a long list :)





DEPLOYMENT PROCESS


Setup

Below pictorial representation shows the kind of environment, roles and machine we have setup for our Sitecore deployment

Sitecore Environments configured in OctopusDeploy
Sitecore Environments Configured in OctopusDeploy



















Below are the role definitions and their purpose (Note: Role in Octopus term is just the textual tag and you can give any friendly name you wish)
  • CM-Master - This is the role to scope our deployment steps for Sitecore package installations only to this machine. As Sitecore package in our build process contains only Sitecore items, they have to be deployed only once. We use the same role to do content publish (not the site content but platform related items such as templates, layouts etc.)
  • CM-Slave - This is just another CM Node and distinguishes itself from the Master
  • CM-Job - We share the same Job server VM across multiple production instances (not exactly how it is shown in the picture above). To be able to do that we run multiple Sitecore instances on the Job Server VM and for each of the Sitecore instance we have a Octopus tentacle service.
  • CD - These are content delivery servers
  • SingleMaster - Where there is a standalone CM / CD server (not in cluster). There are situations such as development / pre-prod servers where its not the similar environment as in higher or production environments
  • OctopusServer - This is Octopus Server itself which is used to execute Database scripts remotely using SQL Management Objects and client library. We did not want to install tentacle on database server.
  • There are few others

Process


We have about 20 step deployment process and some of the steps has child steps within it.

I am putting high level details of important steps but you will get an idea on how to build this



First 4 steps
Step 5


  1. As a first step, we check if the instance is up for us to do the deployment (this is different than tentacle health). We do this using PS command Invoke-WebRequest and checking the status code to match 200 OK.
  2. If the above step successful, we install a TDS service package (to ensure its there - this is converted to nuget package by us and uploaded to Nuget feed) which is needed for Sitecore package installation.
  3. We then notify the recipients, configured in the OctopusDeploy variable (email address of a distribution list), that the deployment has started
  4. We then take the backups, first for CM databases (Master, Core and Web)
  5. Step 5 above takes the file system backup, and this excludes certain directories such as /sitecore /temp /app_data folders 

  6. Sitecore Package Deployment STep which includes Sitecore Items


  7. Then, as shown above, we deploy the Sitecore Packages only on servers which are in Master roles (CM-Master, SingleMaster etc.) and below is how we do it. Package path parameter is the path where Nuget package deployment task has extracted the Sitecore update package.
  8.         
    #Installing sitecore package using TDS service
    & "$TDSInstallationDirectory\Installer\PackageInstaller.exe" -v -sitecoreUrl "http://$SitecoreUrl" -sitecoreDeployFolder $WebsiteFolder -packagePath $Sitecore_Package_Installation_Directory\$contentFolder\$updatePackage
    
    

  9. We do a HTTP health check (hitting a URL and checking status code 200) to ensure that Sitecore instance is still up

  10. Step 7

  11. Next we do a web package deployment using Nuget package deployment step (available in Octopus library) as shown above. We also do a file clean up to delete certain files which are not required on CM Master role e.g configs which are specific to CD or publishing / job server needs to be deleted. Note: The configuration transformation happens at this stage for specific environment where deployment step is being executed.
  12. We have multiple Sitecore packages from different teams so next few steps does the deployment using similar logic stated above (step 7 & 8)
  13. We then publish the specific areas of content tree to the internal publishing target using our custom service
  14. Notify stakeholders via email for them to proceed with sanity testing before we proceed to CD deployments
  15. We have a manual intervention step (available in Octopus) here for someone to manually click a button once sanity testing is complete, to proceed with CD deployments
  16. Once we start CD deployment (post manual intervention and go ahead), a notification is sent to stakeholders that deployment to CD instances has started
  17. First step is to check if CD servers are responding by doing the HTTP health check with expected status code of 200 OK
  18. Take the backup of Pub (publishing target) database
  19. Take the file system backup (excluding list of folders - as mentioned in step 5 above
  20. Publish Sitecore Items by calling our custom service. Octopus task has a parameter which takes values of desired locations to publish from a variable
  21. Once publishing is done, we do a Nuget web package deployment in all CD servers in Rolling Deployment mechanism
  22. Once Nuget package is deployed, we do a clean up using file system clean up task in OctopusDeploy for specific files and configurations not relevant for Content Delivery servers.
  23. Finally do a HTTP health check with expected status code of 200 OK
  24. Notify stakeholders of Deployment completion


Deployment Considerations


Few considerations which may be helpful when you implement your deployment process


  1. Restoration - Since we are taking the backup of databases and file system, restoration can be done on-demand and that process we have kept outside of this process. To handle such scenario, we have automated the restoration part as a separate Octopus project
  2. AppPool Recycle Requests - Ideally you should not need any explicit application pool recycles but sometimes to clear the HTML output cache on priority we may need to have this in place. We created a separate Octopus project to handle this. Project of this nature must be secured and permissions must be granted only to appropriate group of users such as Octopus Administrators.
  3. Pre-requisite deployments - In our engagement, there are lot of other dependencies such as device detection database from 51 degrees or a security roles and permissions package etc., which are relevant whenever we are setting up a new instance or rarely changed and deployed but not as part of regular releases. You can either have these steps executed conditionally (by means of checking if those already exists on the instance) or have a separate project, like we have, to deploy the pre-requisites first before making your releases.
  4. Deployment step generalization - Keep your deployment steps as generic as possible. If parameters for your deployment steps changes from one environment to another, one deployment role to another or from machine to machine, consider using Octopus variables or better even, variable sets (which allows you to group octopus variables logically)
  5. Define Deployment Lifecycle - If you want your releases should get promoted in particular order e.g. From SIT -> QA -> Staging -> Production then define your project life cycle in octopus and configure that to be used in your deployment process

In my next post I will talk about different considerations about handling Configuration Transformation


Happy Deployments !!


No comments:

Post a Comment