We’ve come to the most important part of this series – securing sensitive data when working with the configuration in ASP.NET Core. As software developers, we are responsible for the security of the applications we create, and it should be on top of our priorities list all the time.
If working in a big team or on a big project, handling sensitive information becomes even more important because we can cause problems for other developers in their development environment.
We’ll see a scenario of a potential problem that can happen really easily.
Let’s dive in.
What Counts as Sensitive Data and Why Should We Care
We consider sensitive data to be everything and anything that can potentially be exploited by a third party. For example, API keys, tokens, connection strings, emails, passwords, password hashes, potentially some URLs, and other types of data.
It’s pretty much impossible that you’ve done any serious software development without leaving a sensitive piece of information somewhere in the commit history.
Whether we know it or not.
We know we have, and it’s pretty easy to fall for that trap. You’re working on your side project, a quick little proof of concept that will show off your latest idea. In order to create it, you need access to the database, and you quickly create one in Azure/AWS/GCloud. You put the connection string in appsettings.json quickly, just to test it out. A few hours of development later, you are pretty happy with how your project turned out, and since you don’t want to lose that, you quickly push it to the GitHub.
A few years later, you browse your repositories on the GitHub to see how much you’ve advanced and everything you did over the years. You open the project and find out the committed connection string, server name, password, and everything.
Sounds familiar?
Storing Sensitive Data in the Development Environment
We’ve established that sensitive data should not be stored in configuration files, much less in source code itself.
But what is the proper way to store sensitive data in the development environment?
We can choose either to store our configuration in the environment variables or as a user secret by using the Secret Manager.
Neither of these options is good enough for production since they don’t encrypt our data and it’s stored as a plain text. Nevertheless, we can use them to avoid conflicts while working with other developers on a project because we don’t commit secrets and environment variables. They are user-specific (or environment-specific), and that’s how it should be. That way we avoid accidental commits of sensitive information like API keys, connection strings etc.
It’s a common mistake that happened to everyone once or twice at least. If that happens you often need to bend over backward to fix the mistake, since commit history stays forever.
Sometimes it includes revoking API keys or changing database passwords. That is not cool.
Both user secrets and environment variables are stored by flattening the configuration hierarchical structure.
For example, a connection string looks like this in the appsettings.json file:
"ConnectionStrings": {
"sqlConnection": "server=.\\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
},
To be able to read it from the environment variable, we need to create it by using double underscore “__” separator:
ConnectionStrings__sqlConnection="server=.\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
Secrets are stored as JSON values and we can store it by flattening the structure using the ":", like we access the different sections in the source code:
So our secret connection string will look like this:
{
"ConnectionStrings:sqlConnection": "server=.\\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
}
Environment variables are pretty easy to use, but if you’re developing multiple projects on your development machine, you might soon find out that you have a lot of environment variables stored on your machine.
That’s why user secrets are a recommended way to store sensitive data locally.
Secret Manager Commands
User secrets are project-specific configuration values and thus very convenient for the development environment where we can usually have dozens if not hundreds of different projects.
We use something called “Secret Manager” to enable this mechanism and store our secrets locally.
To start using secrets, we need to enable them first for the concrete project by navigating to the project directory and typing:
dotnet user-secrets init
As a result, the command creates an entry in the csproj file:
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<UserSecretsId>06219bf2-6545-49db-b255-908d1fabd932</UserSecretsId>
</PropertyGroup>
We can also do this in Visual Studio by right-clicking on the project and selecting “Manage User Secrets”.
To create or modify a secret connection string we can use the dotnet user-secrets set command:
dotnet user-secrets set "ConnectionStrings:sqlConnection" "server=.\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
To check if we’ve successfully added it, we can use the list command:
dotnet user-secrets list
Now we can safely remove the value from the appsettings.json file (and/or appsettings.Development.json file).
We can also easily check all our secrets by right-clicking on the project and selecting “Manage User Secrets”. This will open our project-specific secrets.json file:
{
"ConnectionStrings:sqlConnection": "server=.\\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
}
Removing a secret is as easy as setting it:
dotnet user-secrets remove "ConnectionStrings:sqlConnection"
To remove all the secrets we can use the clear command:
dotnet user-secrets clear
If you already have a secrets file you want to import into the existing project, you can do that too by piping it to the set command:
type .\input.json | dotnet user-secrets set
This can help immensely since we don’t need to import all the settings manually.
Of course, using secrets doesn’t change the way we access our configuration, so we can use strongly typed objects to map the configuration.
Pretty convenient, isn’t it?
Managing Configuration Data as Environment Variables
If you still prefer to use environment variables to store your configuration, or you’re working on a legacy project that does so, here’s what you can do.
As we’ve mentioned earlier, to use environment variables as configuration values, we need to flatten the hierarchy with double underscore __.
So let’s see how we can set variables for different platforms:
For Windows in CMD, we can use the set command:
set ConnectionStrings__sqlConnection="server=.\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
To review all the environment variables (user-specific) we can just type set without any arguments. This will list all the variables we’ve set so far.
We can use PowerShell to set the variable too:
[System.Environment]::SetEnvironmentVariable('ConnectionStrings__sqlConnection','server=.\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true',[System.EnvironmentVariableTarget]::User)
This is a way more complicated then using CMD, but it might come in handy if using PowerShell scripts to set the configuration variables.
We can check our variable with:
[System.Environment]::GetEnvironmentVariable('ConnectionStrings__sqlConnection','user')
On Linux, we can set the variable using the export command:
export ConnectionStrings__sqlConnection="server=.\\SQLEXPRESS; database=CodeMazeCommerce; Integrated Security=true"
To list all the variables on Linux we can simply type set, and to remove the variable we can use the unset command:
unset ConnectionStrings__sqlConnection
You can pick your own preferred way to do it, but as we’ve already discussed, user secrets seem like a much more convenient way to do it. We have a dedicated CLI and a good project-specific mechanism to do it.
We can safely remove the connection strings and other sensitive data from our appsettings files.
Conclusion
In this article we’ve talked about how to protect sensitive data and avoid conflict locally and while working on bigger projects with multiple software developers. We’ve discussed the pros and cons of using user secrets and environment variables, and we’ve seen how to use both ways to set configuration values for our project.
In the next part, we’re going to see how to protect our sensitive data in the production environment with Azure Key Vault.
You can find other parts of this series on the ASP.NET Core Web API page.