Tools and Best Practices for Secret Management in .NET

In this article, we will talk about the tools and best practices for secret management in .NET applications.

Every application has values it wishes to keep hidden. One of the critical aspects of security in software development is how we handle secrets. Examples of such data include but are not limited to API keys, connection strings, SSH keys, database passwords, encryption keys, and digital certificates. No rule determines what should be regarded as a secret; it varies depending on project needs and decisions made by developers and companies.

Throughout the article, we will use the example of safeguarding a secret API key for payment integration within an e-commerce platform.

Importance of Secret Management

Secret management comprises tools and techniques that ensure the safe storage and consistent confidentiality of our application secrets. We ought to give secret management the utmost attention due to the constant threat posed by malicious users seeking to compromise the integrity of our application. Unprotected secrets can be modified, granting unauthorized access to critical parts of an application and potentially resulting in a data breach.

Moreover, inadequate protection of app secrets could result in data misuse, denial-of-service attacks, and manipulation of processes causing financial loss. Altogether, these damage the system’s reputation.

In our e-commerce application example, the API key is a gateway to sensitive customer data like personal information, credit card details, and purchase history. Exploiting this information could lead to fraudulent activities. Understanding the significance of safeguarding sensitive data like the API key is part of our responsibility to ensure that the applications we build are functional, efficient, and secure.

To address these vulnerabilities, let us look into the provisions for .NET secret management. These tools ensure that these sensitive pieces of information are not hardcoded into our application’s codebase.

Tools for Secret Management in .NET

It’s highly advisable to avoid storing application secrets directly within the source code because anyone with access to the code base can easily view or misuse these sensitive credentials.

Different stages, from development to deployment and maintenance, have varying security requirements. We consider that each stage comes with varying operational contexts and threats scope. For example, in development, speed often takes priority. So, we usually opt for simpler security measures that cover the basics. While deployment exposes us to a wider audience and network, we need stricter security measures to protect against external threats.

Let’s explore some of the tools for secret management in .NET.

The appsettings.json File

The appsettings.json file is a .NET provision for storing custom configurations of an application such as database connection strings, file paths, etc. This file typically resides in the root directory of a project.

For environment-specific settings, we can create separate files (appsettings.Production.json, appsettings.Development.json, etc). These override the corresponding values in the appsettings.json file based on the environment specified in launchSettings.json.

Note that these environment-specific settings are not the same as environment variables.

Using appsettings.json, we can save the API key of our payment channel:

{
  "Paystack": {
    "ApiKey": "sk_test_7f9a384e2dc15181cab8fb83db4fb332769fedbb"
  },
}

The Paystack section holds our ApiKey and its value.

To access the configuration settings in this file, we need an IConfiguration object:

public class PaymentService
{
    private IConfiguration _configuration;
    private readonly string? _key;
    private PayStackApi _paystackApi;

    public PaymentService(IConfiguration configuration)
    {
        _configuration = configuration;
        _key = _configuration["Paystack:ApiKey"];
        _paystackApi = new PayStackApi(_key);
    }
}

Our PaymentService class is responsible for initiating payments in our e-commerce application. Using dependency injection, we pass the IConfiguration interface to the constructor of PaymentService. Through this instance, we retrieve the value from the configuration for the Paystack API and store it in _key. With our API key, we can now successfully initialize payment.

We can retrieve the configuration value of the API key another way by utilizing the GetValue() method in the Microsoft.Extensions.Configuration namespace:

_key = _configuration.GetValue<string>("Paystack:ApiKey");

Both the indexer and GetValue() approaches serve the same purpose but they differ in their retrieval process. The indexer, _configuration["Paystack:ApiKey"], offers a more concise and direct retrieval. However, if the key doesn’t exist, it returns null or throws an exception for non-nullable types. On the other hand, the GetValue() approach enables a type-safe retrieval and allows us to assign default values.

The appsettings.json file serves as a simple configuration solution. It provides only a basic level of security and insufficiently protects our secrets, and thus falls short of safeguarding our app’s sensitive information. The inherent risk lies in the potential exposure of this file through accidental Git commits where it becomes part of the version history. For this reason, even if we remove it later, there might still be traces left in the commit history.

Environment Variables

Environment variables are set outside an application. They typically hold simple string key-value pairs. They are dynamic and we can manipulate them during runtime through the terminal or command prompt, being tied to the operating environment of a process.

When naming environment variables, instead of a colon (“:”), opt for a single underscore (“_”) as a separator for simple key-value pairs and double underscores (“__”) to indicate nested structures.  This practice prevents issues on systems like Windows, where colons are used in drive paths. Additionally, environment variables face limitations storing complex nested JSON due to string limitations and parsing complexity. Consequently, double underscores simplify the representation of hierarchy within variable names, flattening the structure.

Previously, we entered our API key into appsettings.json. Now let’s define it as an environment variable using Windows PowerShell:

setx Paystack__ApiKey sk_test_7f9a384e2dc15181cab8fb83db4fb332769fedbb

We can retrieve the value using the GetEnvironmentVariable():

_key = Environment.GetEnvironmentVariable("Paystack__ApiKey", EnvironmentVariable.User);

The EnvironmentVariableTarget.User specifies the target location for locating the environment variable. It indicates the currently logged-in user account on a Windows system. However, on systems that don’t employ the Windows registry structure, this targeting won’t function and would result in a null return if utilized.

macOS and Unix-based systems use shell configuration files like .bash_profile, .bashrc, or .profile for user-specific environment variables. These files are in the user’s home directory and are loaded when they log in.

Environment variables take precedence over app settings, potentially overriding shared keys. Despite being more secure, there is still a risk of exposure due to inadequate access controls or shared environments.

Secret Manager

The Secret Manager is a built-in .NET tool that safeguards project-specific app secrets. It stores the secrets locally and unencrypted in a file named secrets.json outside the project tree. This makes it impossible for us to commit the file to Git erroneously. By default, this file is found within a user profile directory.

On Windows, the file path is typically:

%APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json

On macOS and Linux:

~/.microsoft/usersecrets/<user_secrets_id>/secrets.json

To enable Secrets Manager in Visual Studio for Windows, we right-click on the project and select “Manage User Secrets”. However, this method does not apply to other IDEs and operating systems. The .NET CLI allows for an OS- and IDE-agnostic approach. Let’s run the init command in our project directory:

dotnet user-secrets init

With user secrets enabled, a UserSecretsId element is added within the PropertyGroup of our project file:

<PropertyGroup>
  <TargetFramework>net8.0</TargetFramework>
  <Nullable>enable</Nullable>
  <ImplicitUsings>enable</ImplicitUsings>
  <UserSecretsId>67d4959e-0f22-4ad4-a154-9ba994652a11</UserSecretsId>
</PropertyGroup>

To add a secret, we use commands like dotnet user-secrets set :

dotnet user-secrets set "Paystack:ApiKey" "sk_test_7f9a384e2dc15181cab8fb83db4fb332769fedbb"

Just as we saw with the appsettings configuration file, we retrieve our secrets by also using an instance of IConfiguration interface:

_key = _configuration["Paystack:ApiKey"];

Unlike environment variables, using colons in user secrets is not an issue.

Secret Manager seamlessly integrates with .NET’s configuration system but the secrets are not available in production deployments. As a result, it is primarily intended for use during development. In the next section, we will address more secure services that can be used in production.  

More Robust Secret Management Tools

All the secret management provisions we have learned about so far store data without encryption. This presents significant security risks, especially in production environments, where the stakes are high, and the consequences of a security breach can be severe. Therefore, to ensure the safety of sensitive information during production, we must take extra precautions. 

There are two categories of enhanced solutions available for use – cloud-based and non-cloud-based. Some of the popular cloud-based services include Azure Key Vault, AWS Secrets Manager, IBM Cloud Secrets Manager, Google Cloud Secret Manager, and HashiCorp Vault Enterprise (cloud option). On the other hand, non-cloud-based services include HashiCorp Vault Enterprise (non-cloud option), Thycotic Secret Server, KeyWhiz, etc.

With any of the options we listed above, we control access to our secrets using detailed policies and role-based authorizations. These solutions generally encrypt our secrets and provide centralized management and storage for cryptographic keys and secrets. They offer advanced features such as key rotation, and auditing and are highly scalable. 

The process of integrating any of these options into our .NET application may vary depending on the solution provider’s SDKs. However, the high-level steps are quite similar:

• Authentication and access
• SDK/Integration setup
• Settings configuration
• Accessing secrets
• Error handling

These solutions provide excellent services, but it is important to note that most of them require a subscription or payment based on usage. The setup and configuration process can be quite complex and time-consuming. Additionally, cloud-based services require an internet connection and may not function without one. Organizations that choose non-cloud-based options are responsible for maintaining the infrastructure, which can be resource-intensive. Therefore, it is crucial for us to carefully consider our options before making a decision.

Best Practices for Secret Management

Besides using the tools and features we have discussed, there are best practices we should follow to ensure the highest level of protection for our app secrets.

Use of POCO

When we map our secrets to Plain Old CLR Objects (POCO), we encapsulate the sensitive information. It is a great way to manage, maintain, and secure our secrets.

Let’s define Secrets, a POCO that represents our API key:

public class Secrets
{
    public string ApiKey { get; set; }
}

Next, we will introduce a SecretsManager class where we are going to initialize the ApiKey with the value from a configuration file:

public class SecretManager 
{ 
    private IConfiguration _configuration; 
    private readonly string? _key; 
    
    public Secrets GetSecrets 
    { 
        get 
        { 
            return new Secrets 
            { 
                ApiKey = _key 
            }; 
        } 
    }
 
    public SecretManager(IConfiguration configuration) 
    { 
        _configuration = configuration; 
        _key = _configuration["Paystack:ApiKey"]; 
    } 
}

With the GetSecrets property we defined, let’s see how we use it in our code to retrieve the API key:

public class PaymentServiceWithPoco
{
    private SecretManager _secretManager;
    private Secrets _secrets;
    private readonly string? _key;
    private PayStackApi _paystackApi;

    public PaymentServiceWithPoco(IConfiguration configuration, 
        SecretManager secretManager)
    {
        _secretManager = secretManager;
        _secrets = _secretManager.GetSecrets;
        _key = _secrets.ApiKey;
        _paystackApi = new PayStackApi(_key);
    }
}

We create the PaymentServiceWithPoco class that uses the SecretManager to obtain the secrets it needs to initialize the PayStackApi instance with the extracted API key.

We can always add more secrets to our Secrets class and initialize them in SecretManager. This practice consolidates our secrets into a dedicated object and provides a centralized location for managing them.

Git Ignore Sensitive Files

We can proactively prevent configuration files like appsettings.json and .env files from being committed to Git by adding them to a .gitignore file.

.env, short for environment, is a file that contains environment variables in a key-value pair format.

To create a .gitignore file, we need to open the Terminal for Mac and Linux or Git Bash for Windows, navigate to the repository location, and run the command:

touch .gitignore

If the command succeeds, there will be no output.

In the file, we can then add our secret file:

# Ignore appsettings.json file
appsettings.json
.env

GitHub maintains an official list of recommended files to ignore for different operating systems and IDEs.

Dynamic Secrets

Earlier we mentioned that because we prioritize speed in the development phase, we usually employ minimal security measures. Therefore, we cannot rely on these measures when our application is live. This also implies that development app secrets must differ distinctly from production.

To reduce the risk of compromised information, we recommend using dynamic app secrets. Dynamic secrets change frequently and we can enable auto-generation. As a result, if a secret leaks, the likelihood they are still valid is low.

We recommend using an automated approach for this with the added capacity to revoke them when required. For instance, database credentials can be dynamic with services like HashiCorp Vault, AWS Secrets Manager, etc. which can generate passwords and usernames with short expiration periods.

Principle of Least Privilege

The Principle of Least Privilege (PoLP) supports granting only the minimum level of access or permissions that are necessary to perform an intended function. Adopting this principle in software development means developers should not have access to all secrets in the secrets management system.

For the configuration files such as appsettings.json, secrets.json, and even the environment variables, the permission to view or modify configurations can be set based on roles and responsibilities. However, the details on the specific implementation of this principle go beyond the scope of this article.

Conclusion

In conclusion, safeguarding sensitive data is a necessity to ensure the functionality and efficiency of our applications. We have explored various tools and techniques, ranging from fundamental practices like appsettings.json to more advanced solutions such as Azure Key Vault. With emphasis on the significance of adopting these methods and advocating for best practices like employing POCO to encapsulate secrets, this article equips us with a comprehensive understanding and actionable steps to fortify our applications.