C# 如何在 .NET 中下载大文件（通过 HTTP）？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/1078523/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How do I download a large file (via HTTP) in .NET?
提问by Nick Cartwright
I need to download a largefile (2 GB) over HTTP in a C# console application. Problem is, after about 1.2 GB, the application runs out of memory.
我需要在 C# 控制台应用程序中通过 HTTP下载一个大文件 (2 GB)。问题是，在大约 1.2 GB 之后，应用程序内存不足。
Here's the code I'm using:
这是我正在使用的代码：
WebClient request = new WebClient();
request.Credentials = new NetworkCredential(username, password);
byte[] fileData = request.DownloadData(baseURL + fName);
As you can see... I'm reading the file directly into memory. I'm pretty sure I could solve this if I were to read the data back from HTTP in chunks and write it to a file on disk.
如您所见...我正在将文件直接读入内存。如果我要从 HTTP 分块读取数据并将其写入磁盘上的文件，我很确定我可以解决这个问题。
How could I do this?
我怎么能这样做？
采纳答案by Alex Peck
If you use WebClient.DownloadFileyou could save it directly into a file.
如果您使用WebClient.DownloadFile，您可以将其直接保存到文件中。
回答by Richard
You need to get the response stream and then read in blocks, writing each block to a file to allow memory to be reused.
您需要获取响应流，然后分块读取，将每个块写入文件以允许重用内存。
As you have written it, the whole response, all 2GB, needs to be in memory. Even on a 64bit system that will hit the 2GB limit for a single .NET object.
正如您所写，整个响应（全部为 2GB）都需要在内存中。即使在 64 位系统上，单个 .NET 对象的容量也会达到 2GB 的限制。
Update: easier option. Get WebClientto do the work for you: with its DownloadFilemethod which will put the data directly into a file.
更新：更简单的选择。获取WebClient为您做的工作：与它的DownloadFile方法，这将直接把数据放到一个文件中。
回答by John Saunders
The WebClient class is the one for simplified scenarios. Once you get past simple scenarios (and you have), you'll have to fall back a bit and use WebRequest.
WebClient 类是用于简化场景的类。一旦您通过了简单的场景（并且您已经完成了），您将不得不退后一点并使用 WebRequest。
With WebRequest, you'll have access to the response stream, and you'll be able to loop over it, reading a bit and writing a bit, until you're done.
使用 WebRequest，您可以访问响应流，并且可以循环遍历它，读取一点和写入一点，直到完成。
Example:
例子：
public void MyDownloadFile(Uri url, string outputFilePath)
{
const int BUFFER_SIZE = 16 * 1024;
using (var outputFileStream = File.Create(outputFilePath, BUFFER_SIZE))
{
var req = WebRequest.Create(url);
using (var response = req.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
var buffer = new byte[BUFFER_SIZE];
int bytesRead;
do
{
bytesRead = responseStream.Read(buffer, 0, BUFFER_SIZE);
outputFileStream.Write(buffer, 0, bytesRead);
} while (bytesRead > 0);
}
}
}
}
Note that if WebClient.DownloadFile works, then I'd call it the best solution. I wrote the above before the "DownloadFile" answer was posted. I also wrote it way too early in the morning, so a grain of salt (and testing) may be required.
请注意，如果 WebClient.DownloadFile 有效，那么我将其称为最佳解决方案。我在发布“DownloadFile”答案之前写了上面的内容。我也是早上写得太早了，所以可能需要一点点盐（和测试）。
回答by Whuppa
WebClient.OpenRead returns a Stream, just use Read to loop over the contents, so the data is not buffered in memory but can be written in blocks to a file.
WebClient.OpenRead 返回一个 Stream，只需使用 Read 来循环内容，因此数据不会缓存在内存中，而是可以分块写入文件。
回答by qqus
The connection can be interrupted, so it is better to download the file in small chunks.
连接可能会中断，因此最好以小块下载文件。
Akka streams can help download file in small chunks from a System.IO.Stream using multithreading. https://getakka.net/articles/intro/what-is-akka.html
Akka 流可以帮助使用多线程从 System.IO.Stream 以小块形式下载文件。https://getakka.net/articles/intro/what-is-akka.html
The Download method will append the bytes to the file starting with long fileStart. If the file does not exist, fileStart value must be 0.
Download 方法会将字节附加到以 long fileStart 开头的文件中。如果文件不存在，fileStart 值必须为 0。
using Akka.Actor;
using Akka.IO;
using Akka.Streams;
using Akka.Streams.Dsl;
using Akka.Streams.IO;
private static Sink<ByteString, Task<IOResult>> FileSink(string filename)
{
return Flow.Create<ByteString>()
.ToMaterialized(FileIO.ToFile(new FileInfo(filename), FileMode.Append), Keep.Right);
}
private async Task Download(string path, Uri uri, long fileStart)
{
using (var system = ActorSystem.Create("system"))
using (var materializer = system.Materializer())
{
HttpWebRequest request = WebRequest.Create(uri) as HttpWebRequest;
request.AddRange(fileStart);
using (WebResponse response = request.GetResponse())
{
Stream stream = response.GetResponseStream();
await StreamConverters.FromInputStream(() => stream, chunkSize: 1024)
.RunWith(FileSink(path), materializer);
}
}
}
