In this article, we will look at how we create In-Memory Zip files in C#.
Previously, we discussed how to create and read Zip files in .NET in our Working With Zip Files in C#/.NET article, so we recommend taking a look at that to familiarize yourself with working with Zip files.
Let’s delve in.
In-Memory Zip Files
The initial question is, “Why would we need to create an in-memory Zip file?”
To make a Zip file useful, we typically need to store it somewhere. However, storing it on the file system isn’t always necessary. We can alternatively store it in a database or transmit it directly to the customer through methods like a REST API.
If we intend to send it via a REST call, storing it temporarily on the filesystem is unnecessary.
This article will focus on precisely this scenario.
Create In-Memory Zip Files
But before everything else, let us reference the elephant in the room and show how we can create Zip files in memory.
Without further ado, we can show the method for creating Zip files in memory and discuss its inner workings.
With the help of the previously mentioned article, where we have used Stream, we can deduce the use of MemoryStream to create data in memory and thus create a Zip file in memory:
public static MemoryStream Create(string content)
{
var stream = new MemoryStream();
using (var archive = new ZipArchive(stream, ZipArchiveMode.Create, true))
{
var fileName = $"FileInsideZip.txt";
using var inZipFile = archive.CreateEntry(fileName).Open();
using var fileStreamWriter = new StreamWriter(inZipFile);
fileStreamWriter.Write(content);
}
_ = stream.Seek(0, SeekOrigin.Begin);
return stream;
}
To create a Zip file, we need a Stream to store the data. As we want to store the data in-memory, we will use MemoryStream.
As we have seen in the previous article, we use the ZipArchive class to create a Zip file, so we hook the variable of type ZipArchive to a memory stream. After that, we only need to add data to the Zip archive.
But we will discuss a small detail further as we need to move the stream pointer to the beginning of the stream before we return it to the caller.
And that is it. We have a Zip file in memory and can return it to the caller.
Preparing Project for Zip Files in Memory
To set up our REST API, we’ll utilize the webapi template with the minimal API implemented:
dotnet new webapi -o InMemoryZipFilesInNet
To enhance the realism of our examples, we’ll simulate the development of an application that generates a Zip file containing data from various services within our program.
Our application will leverage multiple services. Whenever a caller invokes our endpoint, we’ll create a Zip file with data from each service.
All services will adhere to the same IService interface, enabling us to retrieve the data:
public interface IService
{
public string Name { get; }
public string GetData();
}
We’ll utilize the Name property for the filename within our Zip file and the GetData() method for the file’s content.
Let’s also create three services: FileService, DbService, and TimeService and incorporate them into our dependency injection container:
builder.Services.AddScoped<IService, FileService>(); builder.Services.AddScoped<IService, DbService>(); builder.Services.AddScoped<IService, TimeService>();
We’ll return the Zip file from the file system as a Stream and byte[].
To do that, let’s name the methods according to their intended functionality:
public interface IGetFile
{
string ContentType { get; }
Stream GenerateFileOnFlyReturnStream();
byte[] GenerateFileOnFlyReturnBytes();
}
With the property ContentType, we will set the content type, which will, in our example, be "application/zip."
Create Zip Files in Memory
By now, we have nicely prepared everything to write the code for creating Zip files, which is the heart of our program.
To start everything, we must create a class implementing the IGetFile interface. Then, we can inject our instance of this class via dependency injection, and everything will work.
The GetZipFile Class
It is time to create a GetZipFile class that will implement the IGetFile interface:
public class GetZipFile(IEnumerable<IService> allServices) : IGetFile
{
private readonly IService[] _allServices = allServices.ToArray();
public string ContentType => "application/zip";
public Stream GenerateFileOnFlyReturnStream() =>
GetServicesAsZipStream(_allServices);
public byte[] GenerateFileOnFlyReturnBytes() =>
GetServicesAsZipBytes(_allServices);
}
Here, we define the GetZipFile class, with a primary constructor that accepts an IEnumerable<IService> and will accept our FileService, DbService, and TimeService classes.
All our examples’ content types are the same, so they are set only once.
Helper Method for Generating Zip Stream
Let’s make a helper method called GenerateArchive() for our program. It’ll create a Zip Stream. The type of the Stream isn’t crucial right now; that’s the beauty of it:
private static void GenerateArchive(Stream stream, IService[] services)
{
using var archive = new ZipArchive(stream, ZipArchiveMode.Create, true);
foreach (IService service in services)
{
var name = $"{service.Name}.txt";
var content = service.GetData();
using var inZipFile = archive.CreateEntry(name).Open();
using var fileStreamWriter = new StreamWriter(inZipFile);
fileStreamWriter.Write(content);
}
}
We create a ZipArchive object using the Stream, which we’ll later pass as a parameter to our method.
After that, we call each service and get the filename and the file content from the service. We create a Zip entry and write it in the Zip file. That is all. Now we have a Zip file on the disk.
Generating a Zip File on the Fly
We already know how to create a Zip file via the Stream. Providing a MemoryStream object to our GenerateArchive() method will result in the Zip file being stored in memory:
private static MemoryStream GetServicesAsZipStream(IService[] services)
{
var stream = new MemoryStream();
GenerateArchive(stream, services);
_ = stream.Seek(0, SeekOrigin.Begin);
return stream;
}
Here, we have an even more straightforward method. We do not have to provide a FileStream; we do not have to find a place on the disk. We are just creating and returning a MemoryStream.
This is the method we were searching for, and it has everything. We create our archive in memory, using dynamic data from our services as a realistic example.
There is just one important note. When we write into the MemoryStream, we are moving from the beginning to the end. For every byte we write, we are moving the pointer forward. So, if we want to return that to the caller, we must first traverse to the beginning with our call to the Seek() method; otherwise, our caller will get nothing.
Return the Array of Bytes
We can stop here as we have seen how to create a Zip file in memory and why we would do that. But as mentioned, sending the file over the wire is not the only use case for in-memory Zip files.
Sometimes, we may want to create a Zip file in memory and write it to a database. In general, we can’t write a Stream to a database, but we can easily store raw bytes in one.
This last example will serve as a gateway to other use cases of in-memory Zip files:
private static byte[] GetServicesAsZipBytes(IService[] services)
{
var stream = new MemoryStream();
GenerateArchive(stream, services);
return stream.ToArray();
}
This is almost the simplest method because we took the time and prepared everything so nicely.
It is very similar to the last one, as we are again creating a MemoryStream. This time, though, we are not returning a MemoryStream, but transforming the stream into byte[].
There is no need to traverse to the beginning this time, as we are calling the ToArray() method.
Return In-Memory Zip Files With Rest API
Now that we’ve completed the preparations, our next step is to determine how to transmit a file from our REST endpoint to the caller.
In .NET for minimal APIs, we utilize the Microsoft.AspNetCore.Http.Results class to deliver data from our endpoints:
app.MapGet("/test-file", () =>
Results.File(MemoryZipFile.Create("Test"), "application/zip", "TestFile.zip"));
app.MapGet("/create-in-memory-zip-file", (IGetFile zipFile) =>
Results.File(zipFile.GenerateFileOnFlyReturnStream(), zipFile.ContentType, "GenerateOnFly.zip"));
app.MapGet("/create-in-memory-zip-file-as-byte-array", (IGetFile zipFile) =>
Results.File(zipFile.GenerateFileOnFlyReturnBytes(), zipFile.ContentType, "GenerateOnFlyAsByteArray.zip"));
As you can see, we use the File method from the Results class to return the created zip file to the client.
Return Bigger Files
We usually generate Zip files because we have a lot of data to transfer. In such situations, we should use non-blocking, async code.
The goal is to prepare data using nonblocking code and transfer files using as little memory as possible.
We will not go into all the details as this is not an article about transferring files using REST protocols or async programming, but let’s look at an example:
app.MapGet("/downloading-bigger-file", async (HttpResponse response, IGetFile zipFile) =>
{
var zipStream = await zipFile.GenerateFileOnFlyReturnStreamAsync();
zipStream.Position = 0;
response.ContentType = zipFile.ContentType;
ContentDispositionHeaderValue contentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = "BigFile.zip"
};
response.Headers[HeaderNames.ContentDisposition] = contentDisposition.ToString();
await zipStream.CopyToAsync(response.Body);
});
First, we prepare a Zip file using non-blocking code. While we can generate data in parallel, creating Zip files must be done sequentially since the ZipArchive is not thread-safe!
After we have prepared the data stream, we can copy the Zip stream directly to the output stream of our REST call, as this will use a minimal amount of memory.
By setting the Content-Disposition header to “attachment”, the client knows it should prompt the user to save the content as a file rather than displaying it directly in the browser.
Such a method could significantly reduce the memory footprint on a high-volume server serving larger files, for example.
Additional Considerations for In-Memory Zip Files
The reality of software development often presents challenges that require careful consideration.
A primary concern with generating in-memory Zip files is memory usage. While modern systems typically have large amounts of available memory, very large files can still strain resources.
Creating the file on a disk may be necessary. Additionally, enabling range processing allows the caller to retry downloads or download specific portions of a file, which is crucial for handling large files efficiently.
Another essential thing that we have neglected in our examples is error handling and logging. Because we have neglected that programming part, the examples are small and easy to follow.
But in reality, we have to think about retrying failed I/O operations, logging, and error handling.
Conclusion
Generating Zip files in memory is relatively straightforward using the ZipArchive class with MemoryStream.
However, in cases where files are too large for memory, creating them on disk may be necessary. In such situations, managing disk usage is crucial. Files can be deleted once transmitted, and scripts or batches can be employed to ensure efficient cleanup while avoiding conflicts with ongoing operations.
