Q：MemoryStream closed & not readable

MemoryStream disables reading when returned

9

1

In my program, I am basically reading in a file, doing some processing to it, and then passing it back to the main program as a memorystream, which will be handled by a streamreader. This will all be handled by a class beside my main.

The problem is, when I return the memory stream from my method in another class, the "canread" variable is set to false, and thus causes the streamreader initialization to fail.

Below is an example of the problem happening (though in here I'm writing to the memorystream in the other class, but it still causes the same error when i pass it back.

In the class named "Otherclass":

public static MemoryStream ImportantStreamManipulator()
{
   MemoryStream MemStream = new MemoryStream();

   StreamWriter writer = new StreamWriter(MemStream);
   using (writer)
   {
       //Code that writes stuff to the memorystream via streamwriter

       return MemStream;
   }
}

The function calls in the main program:

MemoryStream MStream = Otherclass.ImportantStreamManipulator();
StreamReader reader = new StreamReader(MStream);

When I put a breakpoint on the "return MemStream", the "CanRead" property is still set to true. Once I step such that it gets back to my main function, and writes the returned value to MStream, the "CanRead" property is set to false. This then causes an exception in StreamReader saying that MStream could not be read (as the property indicated). The data is in the streams buffer as it should be, but I just can't get it out.

How do I set it so that "CanRead" will report true once it is returned to my main? Or am I misunderstanding how MemoryStream works and how would I accomplish what I want to do?

This is the problem:

using (writer)
{
    //Code that writes stuff to the memorystream via streamwriter

    return MemStream;
}

You're closing the writer, which closes the MemoryStream. In this case you don't want to do that... although you do need to flush the writer, and rewind the MemoryStream. Just change your code to:

public static MemoryStream ImportantStreamManipulator()
{
   // Probably add a comment here stating that the lack of using statements
   // is deliberate.
   MemoryStream stream = new MemoryStream();

   StreamWriter writer = new StreamWriter(stream);
   // Code that writes stuff to the memorystream via streamwriter

   writer.Flush();
   stream.Position = 0;
   return stream;
}

The StreamWriter takes ownership of the memory stream and when the using statement ends, the MemoryStream is also closed.

See Is there any way to close a StreamWriter without closing its BaseStream?.

As others have stated, the problem is that the Stream is closed when the StreamWriter is closed. One possible way to deal with this is to return a byte array rather than a MemoryStream. This avoids having potentially long running objects that must be disposed by the garbage collector.

public static void Main()
{
    OutputData(GetData());
}

public static byte[] GetData()
{
    byte[] binaryData = null;

    using (MemoryStream ms = new MemoryStream())
    using (StreamWriter sw = new StreamWriter(ms))
    {
        string data = "My test data is really great!";

        sw.Write(data);
        sw.Flush();

        binaryData = ms.ToArray();
    }

    return binaryData;
}

public static void OutputData(byte[] binaryData)
{
    using (MemoryStream ms = new MemoryStream(binaryData))
    using (StreamReader sr = new StreamReader(ms))
    {
        Console.WriteLine(sr.ReadToEnd());
    }
}

Another method is to copy the Stream to another stream prior to returning. However, this still has the problem that subsequent access to it with a StreamReader will close that stream.

public static void RunSnippet()
{
    OutputData(GetData());
}

public static MemoryStream GetData()
{
    MemoryStream outputStream = new MemoryStream();

    using (MemoryStream ms = new MemoryStream())
    using (StreamWriter sw = new StreamWriter(ms))
    {
        string data = "My test data is really great!";

        sw.Write(data);
        sw.Flush();

        ms.WriteTo(outputStream);
        outputStream.Seek(0, SeekOrigin.Begin);
    }

    return outputStream;
}

public static void OutputData(MemoryStream inputStream)
{
    using (StreamReader sr = new StreamReader(inputStream))
    {
        Console.WriteLine(sr.ReadToEnd());
    }
}   

Is there any way to close a StreamWriter without closing its BaseStream?

My root problem is that when using calls Dispose on a StreamWriter, it also disposes the BaseStream (same problem with Close).

I have a workaround for this, but as you can see, it involves copying the stream. Is there any way to do this without copying the stream?

The purpose of this is to get the contents of a string (originally read from a database) into a stream, so the stream can be read by a third party component.
NB: I cannot change the third party component.

public System.IO.Stream CreateStream(string value)
{
    var baseStream = new System.IO.MemoryStream();
    var baseCopy = new System.IO.MemoryStream();
    using (var writer = new System.IO.StreamWriter(baseStream, System.Text.Encoding.UTF8))
    {
        writer.Write(value);
        writer.Flush();
        baseStream.WriteTo(baseCopy); 
    }
    baseCopy.Seek(0, System.IO.SeekOrigin.Begin);
    return baseCopy;
}

Used as

public void Noddy()
{
    System.IO.Stream myStream = CreateStream("The contents of this string are unimportant");
    My3rdPartyComponent.ReadFromStream(myStream);
}

Ideally I'm looking for an imaginary method called BreakAssociationWithBaseStream, e.g.

public System.IO.Stream CreateStream_Alternate(string value)
{
    var baseStream = new System.IO.MemoryStream();
    using (var writer = new System.IO.StreamWriter(baseStream, System.Text.Encoding.UTF8))
    {
        writer.Write(value);
        writer.Flush();
        writer.BreakAssociationWithBaseStream();
    }
    return baseStream;
}

128

 

If you are using .NET Framework 4.5 or later, there is a StreamWriter overload using which you can ask the base stream to be left open when the writer is closed.

In earlier versions of .NET Framework prior to 4.5, StreamWriter assumes it owns the stream. Options:

• Don't dispose the StreamWriter; just flush it.
• Create a stream wrapper which ignores calls to Close/Dispose but proxies everything else along. I have an implementation of that in MiscUtil, if you want to grab it from there.

.NET 4.5 has a new method for that:

StreamWriter(Stream, Encoding, Int32, Boolean)

public StreamWriter(
    Stream stream,
    Encoding encoding,
    int bufferSize,
    bool leaveOpen
)

Simply don't call Dispose on the StreamWriter. The reason this class is disposable is not because it holds unmanaged resource but to allow the disposal of the stream which itself could hold unmanaged resources. If the life of the underlying stream is handled elsewhere, no need to dispose the writer.

It seems that the 'close' method of StreamWriter also closes and disposes of the stream. So one must flush, but not close or dispose the streamwriter, so it doesn't close the stream, which would do the equivalent of dispose the stream. Way too much "help" from the API here. 

Avoiding dispose of underlying stream

5

4

I'm attempting to mock some file operations. In the "real" object I have:

StreamWriter createFile( string name )
{
  return new StreamWriter( Path.Combine( _outFolder, name ), false, Encoding.UTF8 ) );
}

In the mock object I'd like to have:

StreamWriter createFile( string name )
{
  var ms = new MemoryStream();
  _files.Add( Path.Combine( _outFolder, name ), ms );
  return new StreamWriter( ms, Encoding.UTF8 ) );
}

where _files is a dictionary to store created files for later inspection.

However, when the consumer closes the StreamWriter, it also disposes the MeamoryStream... :-(

Any thoughts on how to pursue this?

If you subclass the MemoryStream, this will work but you have to call ManualDispose method to close the underlying stream.
I´m not sure but I think this object will be garbage-collected when it goes out of scope.

public sealed class ManualMemoryStream : MemoryStream
{
    protected override void Dispose(bool disposing)
    {
    }
    public void ManualDispose()
    {
        base.Dispose(true);
    }
}

Edit:
This is an alternative if you want the MemoryStream to be flushed and ready to be read from top.

public sealed class ManualMemoryStream : MemoryStream
{
    protected override void Dispose(bool disposing)
    {
        Flush();
        Seek(0, SeekOrigin.Begin);
    }
    public void ManualDispose()
    {
        base.Dispose(true);
    }
}

问题：

       [Fact]
        public void TestStreamEncodingOP()
        {
            var text = Resource.AdsPlatformData_OP;
            var bytes = System.Text.Encoding.UTF8.GetBytes(text);
            using var ms = new System.IO.MemoryStream();
            ms.Write(bytes);
            ms.Seek(0, System.IO.SeekOrigin.Begin);
            static void actionCheck(AdsPlatformProxyOPResult r)
            {
                if (!r.OPSucess) throw new Exception($"Operation failed, Code={r.Result}");
            }
            //listDatas为IEnumerable<AdsPlatform>
            var listDatas = StreamDecodeHelper.DecodeStream<AdsPlatform, AdsPlatformProxyOPResult>(ms, nameof(AdsPlatformProxyOPResult.ResultContent), actionCheck).AsEnumerable();
            //Assert.Equal(4, listDatas.Count);

            //ok，listDatas的方法仅可调用一次
            Assert.True(listDatas.Any());
            var mm=listDatas.Select(_ => _);
            //报错，MemoryStream not readable
            Assert.NotNull(mm.FirstOrDefault());
        }


 public static IEnumerable<TResult> DecodeStream<TResult, TOPCode>(this System.IO.Stream stream, string resultNodeName, Action<TOPCode> actionCheckOPCode = null, bool throwOnEmptyResponse = true) where TOPCode : class
        {
            using var streamReader = new System.IO.StreamReader(stream, Encoding.UTF8);
            //其他操作
         }

　　

streamReader 释放的同时，MemoryStream也释放了，所以第二次 调用 listDatas，报错。
=》listDatas 获取后直接转为 list，以后的操作对list进行操作

https://docs.microsoft.com/en-us/dotnet/api/system.io.streamreader?view=netcore-3.1

Remarks

Unless you set the leaveOpen parameter to true, the StreamReader object calls Dispose() on the provided Stream object when StreamReader.Dispose is called.

The buffer size, in number of 16-bit characters, is set by the bufferSize parameter. If bufferSize is less than the minimum allowable size (128 characters), the minimum allowable size is used.

This constructor enables you to change the encoding the first time you read from the StreamReader object. If the detectEncodingFromByteOrderMarks parameter is true, the constructor detects the encoding by looking at the first four bytes of the stream. It automatically recognizes UTF-8, little-endian Unicode, big-endian Unicode, little-endian UTF-32, and big-endian UTF-32 text if the file starts with the appropriate byte order marks. Otherwise, the user-provided encoding is used. See the Encoding.GetPreamble method for more information.

 Note

When reading from a Stream, it is more efficient to use a buffer that is the same size as the internal buffer of the stream.

 Caution

When you compile a set of characters with a particular cultural setting and retrieve those same characters with a different cultural setting, the characters might not be interpreted correctly, and could cause an exception to be thrown.