Are you keeping your secrets secret?

As developers, we are all guilty of leaking sensitive information of our applications and systems more than we would perhaps like to admit. Don't get me wrong, I'm not talking about breaking the NDAs with our clients but about those little connection strings to our databases and keys to our storage accounts where we hold all our information that we want to protect from outside world. As developers, we understand that this information is sensitive and still we check-in those along with the rest of our code base which eventually ends up in our version control systems that are hosted in the cloud somewhere. And we firmly believe that this is going to be fine. This is akin to watching a dreadful accident on NEWS and saying this can't happen to us.

"The Cloud" is all beautiful and powerful but with power comes responsibility. Among all people, we developers should be well aware and feel responsible & accountable for handling the sensitive information of our applications. We must ensure that it doesn't leak under any circumstances which can jeopardize our systems (and also our positions). And believe it or not cloud has made these things easier and we have all the tools we need to make it happen. It is not complicated anymore, you just need to be a bit thoughtful and set up the habit to do this when you are starting your project (or even better put these things if you end up in a project where this is lacking).

Ok, enough talking let's get to the meat of this post. In this blog post, I will create a Web Application (hosted in Microsoft Azure) which lists URIs of blobs from a container in Storage Account (Microsoft Azure). For the sake of simplicity, I'm creating the container and a dummy blob at the runtime. The Web app is a standard ASP.Net Core web app. I will present two implementations of this scenario.

The first implementation would be with the connection string of a Storage account stored in Application Settings of the web application directly. In the second part, I will move that connection string from the web application and keep it somewhere safe and will fetch information from the Storage account as earlier. Both the applications are deployed using ARM Templates and completely automated (You can create these resources manually if you so wish).

Part I

Here is the source code for this part. I will refer to some bits and pieces of this below:

Here is how the setup looks like: 

If you look at the source code (here), in my WebApWithSecrets solution, I have two projects:

deployment project contains the ARM template for provisioning the resources. The ARM Template does the following

• provisions a Storage Account
  
• provisions the web application
• adds 'StorageAccountConnectionString' as a Connection String in the Web Application

You have instructions to deploy this template in GitHub repository at the above URL (if you want to try it out yourself). Once the resources are provisioned, this is how my resource group looks like when I see it in Azure Portal.

If I navigate to my Web Application and look at the Application Settings, this is what I see: a connection string with the name 'StorageAccountConnectionString' 

Once the resources are provisioned and in place, we can push the code to our web application. You can push this in a variety of ways but for now, I will just right-click and publish (WebAppsWithSecrets project (don't do this in production). Once the code is pushed successfully, the web application will open up in my browser (otherwise navigate to the web application). This is how it looks for me (the red box is my custom code that lists the URIs):

At this point, the app is fully functional and there is nothing wrong here. If I navigate to appsettings.json file in my source code, I will find this: a connection string to my storage account which my code picks and fetches content from Storage Account (you will find the code to render these URIs in Index.cshtml.cs file)

I have this details not only to me but this is now checked-in to my source control.

We can all argue that it is not that bad after all since this is hosted in GitHub (or Azure DevOps or whatever the version control system you are using). These are secure systems and only our team members have access to them - so why bother? 

This all holds true until someone gets hold of your GitHub account or you haven't configured the permissions correctly and someone who shouldn't have the detail might get access. Put it simply, we are increasing the attack surface and we might have to deal with some unwanted situations later on. The good news is we don't necessary have to do it this way, there is a better alternative which I will demonstrate in next part.

Part II

Here is the source code for this part. I will refer to some bits and pieces of this below:

The alternative approach could be that we keep such sensitive information in some sort of secured place or vault which is encrypted and secure which we can rely on and our application simply asks for this information from that vault without ever knowing the details itself. We sort of want to delegate the responsibility of handling sensitive information of our application to some other party without us worrying about the implications. By doing this we not only simplify our lives but also gain much more benefits like it would be better to govern these secrets, we would be able to update the keys and connection strings and certificates without changing anything in our application. The good news is Microsoft Azure has exactly such offer called 'Azure KeyVault'. You can read about this more here.

This is how the redesign of our application looks like after introducing the Azure KeyVault: 

The Web Application has information about the KeyVault only and delegates the responsibility of retrieving ConnectionString to Storage Account to the KeyVault.

If you look at the source code (here). In WebApWithourSecrets solution, I have two projects, same as before.

I have updated my ARM template for this. The template now does the following:

• provisions a Storage Account
• provisions the KeyVault
• provisions the web application
• adds 'StorageAccountConnectionString' as a Secret in the KeyVault
  
• sets the name of the KeyVault in application settings

There is one catch, however. You might ask, wait for a second here! Fine, I moved my storage account's connection string to the KeyVault but I still need details of the vault in my web application to read anything from it. Though I moved the secret of storage account my web application now holds even more sensitive information - all the secrets !! I feel you, I'm with you. But don't worry, this is a solved problem. Microsoft quickly realized this and provided a very nice solution for this very problem. They introduced something called 'Managed Service Identity' aka MSI (now being renamed to just Managed identities) a while back. You can read more about this here but in short, it abstracts this detail in such a way that you don't need to keep any ClientId/ClientSecret or Key of the vault in your web application. You simply enable Managed Identity in your web application (it creates an app principle which gets permissions to read from the Vault). Here is a really good premier of MSI if you want to get a deeper understanding of the feature.

If you navigate to Program.cs file in the solution, here is code that hooks up the KeyVault into our application (BuildWebHost method):

public static IWebHost BuildWebHost(string[] args) =>
     WebHost.CreateDefaultBuilder(args)
     //this is where KeyVault magic happens - we are setting up configurations from Azure KeyVault using Managed Service Identity
     //without specifiying any details of the Azure KeyVault itself (except the Url of the vault)
    .ConfigureAppConfiguration((context, config) =>
    {
       var builtConfig = config.Build();
       var keyVaultUrl = $"https://{builtConfig["KeyVaultName"]}.vault.azure.net";

       //this comes with .net core 2.1
       config.AddAzureKeyVault(keyVaultUrl);
                
       //if using 2.0, you should use this apporach
       //AzureServiceTokenProvider - this is the magic piece that makes it seamless to work with MSI
       //var azureServiceTokenProvider = new AzureServiceTokenProvider();
       //var keyVaultClient = new KeyVaultClient(
       //new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
       //config.AddAzureKeyVault(keyVaultUrl, keyVaultClient, new DefaultKeyVaultSecretManager());
       })
       .UseStartup<Startup>()
       .Build();

if you look closely, all information we have about the Azure KeyVault in our web application is the name (or URL if you will) of the KeyVault (KeyVaultName) stored in our appsettings.json (or Application Settings in the published web app). Everything else is managed by MSI for us. 

In ARM template, we enable MSI for the web application with this property: 

"identity": {
     "type": "SystemAssigned"
   }	

This enables this setting for the web application

So, as we did it Part I, let's deploy the template to provision new set of resources for us (You will find instructions in GitHub repository for this). Once resources are provisioned, this is how it looks for me in Azure Portal:

If i navigate to the application settings of my web application, this is what I see: 

Notice, there is no connection string as we had earlier, instead there is an App Settings 'KeyVaultName' which contains the name of the vault. That's the only information this web application has. Let's deploy the code for the web application (like earlier, right-click and publish). After successfull deployment, this is what you should see: 

This looks exactly same as previous web application since we haven't changed the logic. The only thing that was changed in this web app was to read connection string from KeyVault. 

Now let's uncover some more details. If you navigate to the Azure KeyVault resource and then head over to Secrets, you will see this: 

If you pay attention, you don't see anything, instead, it tells you that "You are unauthorized to view these contents." But the web application lists the URIs just fine! Let's go to Access Policies tab, you will see that there is one policy created for a web application: 

This was created by the ARM template after web application was provisioned. This is the section in ARM Template which defines this policy for the web application

"accessPolicies": [
          {
            "tenantId": "[reference(concat(resourceId('Microsoft.Web/sites', variables('webAppName')), '/providers/Microsoft.ManagedIdentity/Identities/default'), '2015-08-31-PREVIEW').tenantId]",
            "objectId": "[reference(concat(resourceId('Microsoft.Web/sites', variables('webAppName')), '/providers/Microsoft.ManagedIdentity/Identities/default'), '2015-08-31-PREVIEW').principalId]",
            "permissions": {
              "keys": [ "all" ],
              "secrets": [ "all" ]
            }
          }
        ],
        "tenantId": null,
        "tenantId": "[reference(concat(resourceId('Microsoft.Web/sites', variables('webAppName')), '/providers/Microsoft.ManagedIdentity/Identities/default'), '2015-08-31-PREVIEW').tenantId]",
        "sku": {
          "name": "Standard",
          "family": "A"
        }

Since at the moment, the only allowed access is for the web application you are not able to see any secrets when you go to Secrets tab. You can give access to your account if you want to see these secrets. Remember to click Save button after adding the policy, otherwise settings will not be saved

now, when I navigate the Secrets again, i'm able to see the Secrets. 

Remember: If you want to use secrets from the KeyVault while development, you will have to assign access policy for your user in Azure KeyVault. Otherwise, you will get access denied exception. 

Conclusion:

So as we saw, it's fairly straightforward to keep sensitive information in Azure KeyVault and configure our Web Application to read these secrets from the Vault. Moreover,  Managed Service Identity feature further facilitates us to keep the information about KeyVault out of our web application. This can be applied to any scenarios - this is not necessarily bound to the web application only. You can keep connection strings, keys, certificates and all sorts of information you don't want to keep in your consumer application.

So, how do you handle your secrets? What is your opinion on this topic? I would love to hear more from you, please leave your comments. 

Cheers

---

---

comments powered by Disqus