Uploading Large Files in ASP.NET Core

In this article, we are going to learn how to handle uploading large files in ASP.NET Core. We are going to talk about why using byte[] or MemoryStream can be problematic and discuss the benefits of using streams instead.

By leveraging streams, we can significantly improve the performance and scalability of our application when handling large files.

Let’s start.

Support Code Maze on Patreon to get rid of ads and get the best discounts on our products!

Advantages of Using Streams over byte[] or MemoryStream for Large File Handling

Streams are generally better than using byte[] or MemoryStream when handling large files for several reasons.

First, streams can significantly reduce memory usage. With byte[] or MemoryStream, the entire file is loaded into memory before processing, potentially causing problems such as performance issues or memory errors, particularly with large files. In contrast, streams process the file in blocks, enabling more efficient memory management.

Secondly, using streams can also improve processing speed by allowing for simultaneous reading and writing of the file. This means the application can start processing the file faster and finish more efficiently.

Finally, using streams also benefits by enabling the direct streaming of large files over the network. Loading the entire file into memory before sending it, which can be slow and inefficient, is not necessary with byte[] or MemoryStream. Streams transmit data in blocks, reducing latency and improving transmission performance.

Enabling Kestrel Support for the Large Files

We need to establish certain prerequisites and configurations before starting the development process.

Firstly, we need to configure Kestrel in the Program.cs file to allow the upload of large files.

builder.WebHost.ConfigureKestrel(serverOptions =>
{
    serverOptions.Limits.MaxRequestBodySize = long.MaxValue;
});

It is worth noting that by setting the MaxRequestBodySize to long.MaxValue, we are allowing the uploading of files of any size. We can configure this according to our needs and requirements. So we should be careful with this.

Also instead of globally increasing the request limit size, we can use the [RequestSizeLimit(<Size in bytes>)] to specifically increase the request size limit for the specific action only.

Next, we need to create a new class called FileUploadSummary to handle the responsibility for the file upload POST request:

public class FileUploadSummary
{
    public int TotalFilesUploaded { get; set; }
    public string TotalSizeUploaded { get; set; }
    public IList<string> FilePaths { get; set; } = new List<string>();
    public IList<string> NotUploadedFiles { get; set; } = new List<string>();
}

We also utilize the MultipartFormDataAttribute action filter to validate the content type of incoming requests:

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
public class MultipartFormDataAttribute : ActionFilterAttribute
{
    public override void OnActionExecuting(ActionExecutingContext context)
    {
        var request = context.HttpContext.Request;

        if (request.HasFormContentType 
            && request.ContentType.StartsWith("multipart/form-data", StringComparison.OrdinalIgnoreCase))
        {
            return;
        }

        context.Result = new StatusCodeResult(StatusCodes.Status415UnsupportedMediaType);
    }
}

This ensures that the requests content-type the header is correct and starts with a multipart/form-data string before executing the Upload method in the FileController or else it will return a 415 Unsupported Media Type  response.

Upload Large Files Using Streams

Before we start modifying our controller, we have to add one more attribute to our solution:

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter
{
    public void OnResourceExecuting(ResourceExecutingContext context)
    {
        var factories = context.ValueProviderFactories;
        factories.RemoveType<FormValueProviderFactory>();
        factories.RemoveType<FormFileValueProviderFactory>();
        factories.RemoveType<JQueryFormValueProviderFactory>();
    }

    public void OnResourceExecuted(ResourceExecutedContext context)
    {
    }
}

We have to disable the model validation because we don’t want model binders to inspect the body and fetch it into memory or onto disk. This would beat the purpose of using streams.

Now, let’s take a look at controller action that uses streams to upload large files:

[HttpPost("upload-stream-multipartreader")]
[ProducesResponseType(StatusCodes.Status201Created)]
[ProducesResponseType(StatusCodes.Status415UnsupportedMediaType)]
[MultipartFormData]
[DisableFormValueModelBinding]
public async Task<IActionResult> Upload()
{
    var fileUploadSummary = await _fileService.UploadFileAsync(HttpContext.Request.Body, Request.ContentType);

    return CreatedAtAction(nameof(Upload), fileUploadSummary);
}

Here, we have an action method with an HTTP POST attribute that specifies the upload endpoint. It also has a custom action filter attribute MultipartFormData which ensures that the incoming request has the correct content type of multipart/form-data. Of course, we call the DisableFormValueModelBinding filter, to disable model validation.

The method uses the UploadFileAsync method from the _fileService service to upload the file. The UploadFileAsync method takes in the request body stream and content type, which are obtained from the HttpContext.Request.Body and Request.ContentType properties respectively.

Finally, the method returns a 201 Created result with an fileUploadSummary object that contains information about the uploaded file.

 Now, let’s take a look at how we can further improve this implementation by introducing a FileService class. This class provides a service method that reads files from the request body streams and saves them to a folder:

public async Task<FileUploadSummary> UploadFileAsync(Stream fileStream, string contentType)
{
    var fileCount = 0;
    long totalSizeInBytes = 0;

    var boundary = GetBoundary(MediaTypeHeaderValue.Parse(contentType));
    var multipartReader = new MultipartReader(boundary, fileStream);
    var section = await multipartReader.ReadNextSectionAsync();

    var filePaths = new List<string>();
    var notUploadedFiles = new List<string>();
    
    while (section != null)
    {
        var fileSection = section.AsFileSection();
        if (fileSection != null)
        {
            totalSizeInBytes += await SaveFileAsync(fileSection, filePaths, notUploadedFiles);
            fileCount++;
        }

        section = await multipartReader.ReadNextSectionAsync();
    }

    return new FileUploadSummary
    {
        TotalFilesUploaded = fileCount,
        TotalSizeUploaded = ConvertSizeToString(totalSizeInBytes),
        FilePaths = filePaths,
        NotUploadedFiles = notUploadedFiles
    };
}

The UploadFileAsync method uploads files to the server and returns a summary of the upload. It reads the multipart data from the input stream and saves each file to disk. It also calculates the total size of the uploaded files and the number of files uploaded. The FileUploadSummary includes the total size and number of uploaded files, the paths to the uploaded files, and a list of files that could not be uploaded.

You can check the entire code of the service using our GitHub repo.

Uploading Large Files with Postman

Now, let’s take a look at how we can send files using Postman:

We need to select the form-data option and add two key-value pairs where the key will be the name of the file, and the value will be the file itself. We also need to set the Content-Type header to "multipart/form-data; boundary=some value" the header tab in Postman.

Here we have the server response:

{
    "totalFilesUploaded": 3,
    "totalSizeUploaded": "5.93MB",
    "filePaths": [
        "path_to_file\\Image.png",
        "path_to_file\\FilesUploaded\\Zip.zip"
    ],
    "notUploadedFiles": [
        "ImageNotAllowed.tiff"
    ]
}

The response shows the total number of files uploaded, the total size of the files, and the paths of the uploaded files. Additionally, there is a list of files that were not uploaded due to not being allowed.

And here we have the response we would receive if we try to send a POST request with a Content-Type other than multipart/form-data:

{
    "type": "https://tools.ietf.org/html/rfc7231#section-6.5.13",
    "title": "Unsupported Media Type",
    "status": 415,
    "traceId": "00-eade1095704664c0559a58157b192b0c-a6b99260dc6e82ce-00"
}

The response includes a 415 Unsupported Media Type status code along with a traceId to help with debugging.

Conclusion

Uploading large files can be a challenging task, but utilizing streams can greatly simplify the process. With streams, we can efficiently read and write data in small chunks, minimizing memory usage and allowing us to handle large files without performance issues.

By implementing the techniques and strategies we have covered in this article, we can now confidently handle large file uploads in our ASP.NET Core applications.

Liked it? Take a second to support Code Maze on Patreon and get the ad free reading experience!