Thursday, November 23, 2017

Deploying Sitecore 9 to Azure PaaS with Sitecore Azure Toolkit

While there is bunch of documentation available to get started with Sitecore deployment on Azure PaaS, I am sharing my experience of deploying a simple Sitecore 9 deployment topology (XPSingle).

For you to be able to deploy Sitecore 9 on Azure PaaS you need few pre-requisites. These are well documented by Sitecore and you can refer it in the Sitecore Azure Toolkit documentation.

Since most of the pre-requisites were in place for me here are few things you need to get started

Sitecore Azure Toolkit 2.0 


  • Download the zip, Unblock it, and unzip it in an appropriate place in your file system.
  • You can import the Powershell module from Sitecore.Cloud.Cmdlets.psm1 in tools directory and explore the command-lets


Sitecore 9 Web deployment packages


  • Based on the topology you want to deploy, download the web deploy packages (scwdp files) from Sitecore download portal. In my case it is Sitecore 9 XPSingle and I downloaded two packages (one for sitecore - Sitecore 9.0.0 rev. 171002 (Cloud)_single.scwdp.zip and another is for xConnect  - Sitecore 9.0.0 rev. 171002 (Cloud)_xp0xconnect.scwdp.zip)
  • As we will need these packages accessible over internet, upload these packages in an Azure blob storage account under container named based on your convenience. Please review the type of container and your access requirements. In my case, since I was experimenting, I kept it publicly accessible.


ARM templates


  • Download azuredeploy.parameters.json template from github.
  • I preferred the released version Sitecore Azure Quickstart Templates 2.0 (https://github.com/Sitecore/Sitecore-Azure-Quickstart-Templates/releases), but you may want to refer the latest from the repository
  • Sitecore Azure Toolkit commandlets passes parameters to Azure ARM template to provision your infrastructure and hence you need to have only the azuredeploy.parameters.json file on your local machine.
  • As a best practice, I would suggest you to deploy your ARM templates in your own repository OR azure storage which is accessible over internet as this will not have any impact of changes done on git and you are working against a stable template.


Certificate

Create a pkcs 12 self-signed certificate file as give below (use Powershell command prompt in administrative mode - for some reason powershell ISE did not work for me).

Please note: I did it for on-premise deployment of Sitecore 9 to secure Solr instance but same file would do if you want to avoid creating new one.

keytool
-genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret
-storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext
SAN=DNS:localhost,IP:127.0.0.1 -dname "CN=localhost, OU=Organizational
Unit, O=Organization, L=Location, ST=State, C=Country"


keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12

Now since this is a self-signed certificate and not from a trusted auhority, you also need to inclued an additional parameter in azuredeploy.parameters.json file as below

    "allowInvalidClientCertificates": {
      "value": true
    }

You can also refer the whole walkthrough and more details of the entire deployment process here -> Walkthrough: Deploying a new Sitecore environment to the Microsoft Azure App service

Deployment Script

Once you have everything in place, create a new Powershell script in Windows Powershell ISE (in administrative mode). Replace values with you own setup (refer comments for each line) and start deploying

##Path to Sitecore Azure Toolkit commandlet script that you will be importing

$SCAzToolkitPath = "C:\Sitecore9\Azure\Sitecore Azure Toolkit 2.0.0 rev.171010\tools\Sitecore.Cloud.Cmdlets.psm1"

##this is the name used for creating a resource group on azure and also will be a prefix for lot of services.
##Keep it small case with letters and numbers only. Follow azure resource naming rules.
$DeploymentId = "sc9xpsingle"

##Certificate .p12 file created using keytool. Replace it with your physical location
$CertFile = "C:\Sitecore9\Azure\Certificate\solr-ssl.keystore.p12"

##Path to your license file
$LicenseXmlPath = "C:\MyWork\Sitecore\License\license.xml"

##Deployment region - Refer Sitecore Azure deployment compatibility matrix here https://kb.sitecore.net/articles/617478
$Location = "East US"

##Your azure subscription ID, copy it from your azure portal by navigating it to subscription and selecting appropriate subscription under which you want to deploy your instance
$SubscriptionId = "< your subscription ID >"

##This has to be internet addreseable URL otherwise you face errors as shown in gotcha 1.
$ARMTemplate = "https://<storageaccount>.blob.core.windows.net/packages/Templates/XPSingle/azuredeploy.json"

##This is a physical (local) path
$ARMParametersTemplate = "C:\Sitecore9\Azure\Sitecore-Azure-Quickstart-Templates-2.0\Sitecore-Azure-Quickstart-Templates-2.0\Sitecore 9.0.0\XPSingle\azuredeploy.parameters.json"

##Set the parameters that needs to be passed to the ARM templates
##Please note that I have uploaded the deploy packages on azure storage at a particular location, your location may vary

$Parameters = @{
    "authCertificateBlob" = [System.Convert]::ToBase64String([System.IO.File]::ReadAllBytes($CertFile));
    "singleMSDeployPackageUrl" = "https://<storageaccount>.blob.core.windows.net/packages/xpsingle/Sitecore%209.0.0%20rev.%20171002%20(Cloud)_single.scwdp.zip";
    "xcSingleMsDeployPackageUrl" = "https://<storageaccount>.blob.core.windows.net/packages/xpsingle/Sitecore%209.0.0%20rev.%20171002%20(Cloud)_xp0xconnect.scwdp.zip";
    "authCertificatePassword" = "secret";
    "sitecoreAdminPassword" = "Passw0rd";
    "sqlServerLogin" = "badal";
    "sqlServerPassword" = "Passw0rd"; ##ensure you have the password according to SQL Azure password policy -> https://docs.microsoft.com/en-us/sql/relational-databases/security/password-policy
    "location" = $Location;
    "deploymentId" = $DeploymentId
}

##Import your Sitecore Azure Toolkit commandlets
Import-Module -verbose $SCAzToolkitPath

Add-AzureRmAccount ##Add-AzureAccount will not work

Set-AzureRmContext -SubscriptionId $SubscriptionId

##make sure you use -verbose to see the output of deployment status as it happens
Start-SitecoreAzureDeployment -verbose -Location $Location `
                                -LicenseXmlPath $LicenseXmlPath `
                                -SetKeyValue $Parameters `
                                -Name $DeploymentId `
                                -ArmTemplatePath $ARMTemplate `
                                -ArmParametersPath $ARMParametersTemplate


Deployment takes about 20-30 mins. Once your deployment is successful, you can login to sitecore as you would normally on your local deployment. In Azure, It deployes about 19 resources (app service plan, Azure App, Azure Search, SQL Databases, Redis Cache and bunch of others) as shown below

Sitecore Azure Resources for XP Single deployment topology



Few Gotchas / Errors I faced


1. Phyical path Vs URL for your azure template
I had given the physical path of my azuredeploy.json template and azure could not identify the nested templates during deployment. I got this error below and I changed it to the internet addressable path

'The template resource 'templateLinkBase' at

line '28' and column '30' is not valid: The language expression property 'templateLink' doesn't

exist, available properties are 'template, parameters, mode, provisioningState'


2. When I changed the path, I also changed the path of the parameters template to internet addressable path and i got this error. Hence you need to have the parameters file (azuredeploy.parameters.json) on your local machine

Start-SitecoreAzureDeployment : A parameter cannot be found that matches parameter name 'Raw'.
At C:\Sitecore9\Azure\DeploymentScript\xpSingleDeploy.ps1:27 char:1
+ Start-SitecoreAzureDeployment -verbose -Location $Location `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Start-SitecoreA 
   zureDeployment
3. Ensure you execute your Start-SitecoreAzureDeployment commandlet with -verbose to get enough clues on whats going wrong

4. Also ensure you set the parameters key value pair in @parameter variable for deploymentId, and location as this did not seem to be passed though the command (as you can see I have done it at both places - especially for location). Also check other paramters as shown in the script above

5. I also gave deploymentID as something with capital letters, which was not accepted as a valid resource names for Azure hence keep it small case.

Tuesday, November 14, 2017

Federated Authentication for Sitecore 9 integrating with Azure AD - Step by Step

I started integrating Sitecore 9 with Azure AD and I ended up at two resources (in fact 3, but only 2 public sources, 3rd one was only accessible to people who were registered for Sitecore 9 early access program)

First: Sitecore documentation:

Second:
Bas Lijten blog on enabling the federated authentication with Auth0 helped a lot. and he has also added some sample code in the early access program forum.

But I thought most likely, enterprises would like to integrate with Azure AD for following reasons
  1. Relatively widely used Identity / Federated identity provider
  2. Allows you to sync with your enterprise active directory
  3. And allows you to federate with other organizations given the current era of digital landscape where multiple agencies are involved in your brand story e.g. Technology partners, infrastructure partners, creative agencies and many more.

Note:
  • Sitecore 9 uses ASP.NET Identity and OWIN middleware. Sitecore.Owin and Sitecore.Owin.Authentication are the libraries implemented on top of Microsoft.Owin middleware and supports OpenIDConnect out of the box, with little bit of code you need to add yourself :)
  • The scenario I am covering here is for CM environment. For CD environments it should be pretty straight forward.

So here are the steps to get it working, with some gotchas to keep in mind while troubleshooting

Create an Azure AD service in your azure subscription as shown below


Optional:  Map your AD to your custom domain and then create a TXT record on your domain service provider with the properties as shown below and then verify that on Azure


Register an application on Azure AD through "App Regitration" as shown below. xp0.sc in this case is my sitecore instance name I created locally. Also note the Home page URL


Also configure the ReplyURL where the token will be posted to the same URL as Home page.

Create a few groups representing your sitecore roles e.g. Developer, Administrator, Content Author etc.. (Please take note of the Obeject ID shown when you create these groups as this will be needed in the configuration later for transforming these into Sitecore roles)



Create few users and add different users to different groups

Edit the application manifest file to change the default groupMembershipClaims property from null to SecurityGroup / All. This is needed to send the group claims as part of the token. I struggled to get users log in into Sitecore despite of being authenticated by AD as it doesnt have any group claim and as a result the transformation to convert them into Sitecore roles will not kick-in and Sitecore will prompt saying you do not have appropriate accesses to login. Screenshot of the manifest is shown below and this can be edited by clicking on the newly registered app (xp0.sc in my case) and then clicking on Manifest (as shown below)



At this point your Azure AD instance is setup to authenticate users. Next thing we need to do is integrate it with Sitecore and for that we need to add a patch configuration and a custom processor.

Create a visual studio project by creating a new MVC Empty project and remove all un-necessary files and folders, global.asax, app_data, app_start and web.config file (highlighted below)




Add a nuget source for Sitecore using Tools - > Nuget Package Manager -> Package Manager Settings as shown below


Add web.config from your Sitecore instance into the visual studio solution

Create a publish profile (file system) to publish your project output into Sitecore root folder

Exclude files to be deployed as part of the publish in the custom publish profile you created , as indicated below
<?xml version="1.0" encoding="utf-8"?>
<!--
This file is used by the publish/package process of your Web project. You can customize the behavior of this process
by editing this MSBuild file. In order to learn more about this please visit https://go.microsoft.com/fwlink/?LinkID=208121. 
-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <WebPublishMethod>FileSystem</WebPublishMethod>
    <LastUsedBuildConfiguration>Debug</LastUsedBuildConfiguration>
    <LastUsedPlatform>Any CPU</LastUsedPlatform>
    <SiteUrlToLaunchAfterPublish />
    <LaunchSiteAfterPublish>True</LaunchSiteAfterPublish>
    <ExcludeApp_Data>False</ExcludeApp_Data>
    <publishUrl>C:\inetpub\wwwroot\xp0.sc</publishUrl>
    <DeleteExistingFiles>False</DeleteExistingFiles>
    <ExcludeFilesFromDeployment>bin\Sitecore.Kernel.dll;bin\Sitecore.Mvc.dll;bin\Sitecore.Mvc.Presentation.dll;bin\Sitecore.Mvc.Analytics.dll;web.config</ExcludeFilesFromDeployment>
  </PropertyGroup>
</Project>
Once your AD instance is setup, and your solution is created, you can start integrating it with Sitecore and the first step would be to enable Sitecore.Owin.Authentication.Enabler.config file which is available in App_Config\Include\Examples.

Please note, all your custom config files must go in Include folder and you should not be changing anything in Sitecore layer (layered configuration). Below is the snapshot of my solution view where configuration files are created

Also add a custom patch configuration file to include your other federated authentication configurations. I will explain each bit of it in inline comments in the configuration below
<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/">
  <sitecore role:require="Standalone or ContentDelivery or ContentManagement">
    <settings>
      <!-- Below settings describes Azure AD details to which we are integrating with -->
      <setting name="ClientId" value="application ID when you register your application into Azure through APP registration" />
      <setting name="AADInstance" value="https://login.microsoftonline.com/{0}" />
      <setting name="Tenant" value="your azure active directory e.g. badalkotecha.onmicrosoft.com" />
      <setting name="PostLogoutRedirectURI" value="https://xp0.sc/sitecore/login" />
      <setting name="RedirectURI" value="https://xp0.sc/sitecore" />
    </settings>
    <pipelines>
      <owin.identityProviders>
        <!-- This is the custom processor that gets executed when azure AD posts the token to Sitecore -->
        <processor type="SitecoreFedAuthWithAzureAD.Pipelines.CustomAzureADIdentityProvider, SitecoreFedAuthWithAzureAD" resolve="true" />
      </owin.identityProviders>
    </pipelines>
    <federatedAuthentication>
      <identityProviders hin="list:AddIdentityProvider">
        <identityProvider id="xp0.sc.azureAD" type="Sitecore.Owin.Authentication.Configuration.DefaultIdentityProvider, Sitecore.Owin.Authentication">
          <param desc="name">$(id)</param>
          <param desc="domainManager" type="Sitecore.Abstractions.BaseDomainManager" resolve="true" />
          <caption>Sign-in with Azure Active Directory</caption>
          <domain>sitecore</domain>
          <icon>/sitecore/shell/themes/standard/Images/24x24/msazure.png</icon>
          <transformations hint="list:AddTransformation">
            <!-- you need to have and Idp Claim for this to work -->
            <transformation name="Idp Claim" ref="federatedAuthentication/sharedTransformations/setIdpClaim" />
            <!-- This is to transform Azure group into Sitecore Role. The claim value below is the object id that needs to be copied from Azure -->
            <transformation name="Transform to Sitecore DEV Role" type="Sitecore.Owin.Authentication.Services.DefaultTransformation, Sitecore.Owin.Authentication">
              <sources hint="raw:AddSource">
                <claim name="groups" value="2656b61c-748a-4d52-904c-099044c4dcd5" />
              </sources>
              <targets hint="raw:AddTarget">
                <claim name="http://schemas.microsoft.com/ws/2008/06/identity/claims/role" value="Sitecore\Developer" />
              </targets>
              <keepSource>true</keepSource>
            </transformation>
          </transformations>
        </identityProvider>
      </identityProviders>
      <!-- Property initializer assigns claim values to sitecore user properties -->
      <propertyInitializer type="Sitecore.Owin.Authentication.Services.PropertyInitializer, Sitecore.Owin.Authentication">
        <maps hint="list">
          <map name="email claim" type="Sitecore.Owin.Authentication.Services.DefaultClaimToPropertyMapper, Sitecore.Owin.Authentication">
            <data hint="raw:AddData">
              <!--claim name-->
              <source name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" />
              <!--property name-->
              <target name="Email" />
            </data>
          </map>
          <map name="Name claim" type="Sitecore.Owin.Authentication.Services.DefaultClaimToPropertyMapper, Sitecore.Owin.Authentication">
            <data hint="raw:AddData">
              <!--claim name-->
              <source name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname" />
              <!--property name-->
              <target name="Name" />
            </data>
          </map>
        </maps>
      </propertyInitializer>
      <identityProvidersPerSites>
        <mapEntry name="all" type="Sitecore.Owin.Authentication.Collections.IdentityProvidersPerSitesMapEntry, Sitecore.Owin.Authentication">
          <sites hint="list">
            <site>shell</site>
            <site>login</site>
            <site>admin</site>
            <site>service</site>
            <site>modules_shell</site>
            <site>modules_website</site>
            <site>website</site>
            <site>scheduler</site>
            <site>system</site>
            <site>publisher</site>
          </sites>
          <!-- Registered identity providers for above providers -->
          <identityProviders hint="list:AddIdentityProvider">
            <identityProvider ref="federatedAuthentication/identityProviders/identityProvider[@id='xp0.sc.azureAD']" />
          </identityProviders>
          <!-- ExternalUserBuilder is what creates a user with customusername in Sitecore and assigns roles based on claim transformation configured above -->
          <externalUserBuilder type="Sitecore.Owin.Authentication.Services.DefaultExternalUserBuilder, Sitecore.Owin.Authentication">
            <param desc="isPersistentUser">true</param>
          </externalUserBuilder>
        </mapEntry>
      </identityProvidersPerSites>
    </federatedAuthentication>
  </sitecore>
</configuration>
Add below nuget package references for us to use OpenIDConnect as the identity provider
Microsoft.Owin.Security.OpenIdConnect
Sitecore.Owin
Sitecore.Owin.Authentication

Add code to process your authentication events

using Microsoft.Owin.Security;
using Microsoft.Owin.Security.OpenIdConnect;
using Owin;
using Sitecore;
using Sitecore.Configuration;
using Sitecore.Diagnostics;
using Sitecore.Owin.Authentication.Configuration;
using Sitecore.Owin.Authentication.Pipelines.IdentityProviders;
using Sitecore.Owin.Authentication.Services;
using System.Globalization;
using System.Threading.Tasks;

namespace SitecoreFedAuthWithAzureAD.Pipelines
{
    public class CustomAzureADIdentityProvider : IdentityProvidersProcessor
    {

        public CustomAzureADIdentityProvider(FederatedAuthenticationConfiguration federatedAuthenticationConfiguration) : base(federatedAuthenticationConfiguration)
        {
        }

        protected override string IdentityProviderName => "xp0.sc.azureAD";

        protected override void ProcessCore([NotNull] IdentityProvidersArgs args)
        {
            Assert.ArgumentNotNull(args, nameof(args));

            var identityProvider = this.GetIdentityProvider();
            var authenticationType = this.GetAuthenticationType();

            string aadInstance = Settings.GetSetting("AADInstance");
            string tenant = Settings.GetSetting("Tenant");
            string clientId = Settings.GetSetting("ClientId");
            string postLogoutRedirectURI = Settings.GetSetting("PostLogoutRedirectURI");
            string redirectURI = Settings.GetSetting("RedirectURI");

            string authority = string.Format(CultureInfo.InvariantCulture, aadInstance, tenant);

            args.App.UseOpenIdConnectAuthentication(new OpenIdConnectAuthenticationOptions
            {
                Caption = identityProvider.Caption,
                AuthenticationType = authenticationType,
                AuthenticationMode = AuthenticationMode.Passive,
                ClientId = clientId,
                Authority = authority,
                PostLogoutRedirectUri = postLogoutRedirectURI,
                RedirectUri = redirectURI,

                Notifications = new OpenIdConnectAuthenticationNotifications
                {
                    SecurityTokenValidated = notification =>
                    {
                        var identity = notification.AuthenticationTicket.Identity;

                        foreach (var claimTransformationService in identityProvider.Transformations)
                        {
                            claimTransformationService.Transform(identity, new TransformationContext
                            {
                                IdentityProvider = identityProvider
                            });
                        }

                        notification.AuthenticationTicket = new AuthenticationTicket(identity, notification.AuthenticationTicket.Properties);

                        return Task.CompletedTask;
                    }

                }
            });
        }
    }
}


Few Gotchas:
  • Make sure you are not logged in into azure portal as that also uses the azure ad single sign on and the moment you click on federated sign in button in Sitecore, it will take your current session cookie with azure ad and return claims for that user without even asking you to enter credentials. I therefore used incognito window always during debugging.
  • Also note that it does not use the credentials entered in the default sitecore login screen.
  • When you are experimenting with Azure AD always use incognito mode (as mentioned above)
  • Sitecore logout does not kill session with azure ad at this point based on my experiment, but you can extend this pipeline. I have raised  ticket with Sitecore on this.
  • Also ensure that you edit application manifest for your registered application with AD to send groupClaims as indicated in my blog. In the absense of that and the transformation rule for group claim to Sitecore role, you will not be able to login to Sitecore