HighTechTalks DotNet Forums  

XmlSerialization Object Memory Usage

Dotnet Framework (Performance) microsoft.public.dotnet.framework.performance


Discuss XmlSerialization Object Memory Usage in the Dotnet Framework (Performance) forum.



Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old   
Michael Cline
 
Posts: n/a

Default XmlSerialization Object Memory Usage - 10-11-2006 , 08:50 AM






Framework 1.1
Our environment requires us to deserialize xml messages into objects.

In doing some memory analysis I've found the following that doesn't make any
sense to me so I was hoping someone could shed some light on the issue.

Simple Object contains a string, int and bool, as well as another complex
object that also contains a string, int and bool.

If I use code to loop through and create identical objects and add them to a
Hashtable, I see each object taking 430K

If I put a method within that object that 'clones' it and returns a new
object, the new object is only 234K.

Changing the scenario to using the XmlSerializer to deserialize the object
from a string, my memory size of a single object grows to 1016K.

If I call my 'clone' method on this object the size is reduced to 488K.

In my simple mind all of these instances are identical, however there are 4
different sizes which cover a large range.

I'm thinking of creating a CodeGeneration application that could create my
'clone' methods for me, but wanted to make sure that what I was seeing had
some logical justification behind it.

Also, does this behavior change in .NET 2.0 or 3.0? Is there any way for me
to get my 'deserialized' version of the object to be down to the 234K size?

Thanks

Reply With Quote
  #2  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:28 PM






Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

So
Hashtable Storage
for( int i = 0; i < 1000; i++ )
{
ComplexObj Obj1 = new ComplexObj();
Storage.Add( i, Obj1 );
}

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

It's a console app that creates all this and then waits. I show heap with
CLR Profiler and that is where I'm getting the measurements. Do you have a
better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

Reply With Quote
  #3  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:28 PM



Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

So
Hashtable Storage
for( int i = 0; i < 1000; i++ )
{
ComplexObj Obj1 = new ComplexObj();
Storage.Add( i, Obj1 );
}

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

It's a console app that creates all this and then waits. I show heap with
CLR Profiler and that is where I'm getting the measurements. Do you have a
better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

Reply With Quote
  #4  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:29 PM



Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

So
Hashtable Storage
for( int i = 0; i < 1000; i++ )
{
ComplexObj Obj1 = new ComplexObj();
Storage.Add( i, Obj1 );
}

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

It's a console app that creates all this and then waits. I show heap with
CLR Profiler and that is where I'm getting the measurements. Do you have a
better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

Reply With Quote
  #5  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:29 PM



Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

So
Hashtable Storage
for( int i = 0; i < 1000; i++ )
{
ComplexObj Obj1 = new ComplexObj();
Storage.Add( i, Obj1 );
}

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

It's a console app that creates all this and then waits. I show heap with
CLR Profiler and that is where I'm getting the measurements. Do you have a
better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

"Dave Hiniker - MSFT" wrote:

Quote:
Could you clarify the contents of the object (what makes it hundreds of K)?
And how are you measuring the memory overhead in each case? Thanks!

"Michael Cline" wrote:

Framework 1.1
Our environment requires us to deserialize xml messages into objects.

In doing some memory analysis I've found the following that doesn't make any
sense to me so I was hoping someone could shed some light on the issue.

Simple Object contains a string, int and bool, as well as another complex
object that also contains a string, int and bool.

If I use code to loop through and create identical objects and add them to a
Hashtable, I see each object taking 430K

If I put a method within that object that 'clones' it and returns a new
object, the new object is only 234K.

Changing the scenario to using the XmlSerializer to deserialize the object
from a string, my memory size of a single object grows to 1016K.

If I call my 'clone' method on this object the size is reduced to 488K.

In my simple mind all of these instances are identical, however there are 4
different sizes which cover a large range.

I'm thinking of creating a CodeGeneration application that could create my
'clone' methods for me, but wanted to make sure that what I was seeing had
some logical justification behind it.

Also, does this behavior change in .NET 2.0 or 3.0? Is there any way for me
to get my 'deserialized' version of the object to be down to the 234K size?

Thanks

Reply With Quote
  #6  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:29 PM



Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

So
Hashtable Storage
for( int i = 0; i < 1000; i++ )
{
ComplexObj Obj1 = new ComplexObj();
Storage.Add( i, Obj1 );
}

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

It's a console app that creates all this and then waits. I show heap with
CLR Profiler and that is where I'm getting the measurements. Do you have a
better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

"Dave Hiniker - MSFT" wrote:

Quote:
Could you clarify the contents of the object (what makes it hundreds of K)?
And how are you measuring the memory overhead in each case? Thanks!

"Michael Cline" wrote:

Framework 1.1
Our environment requires us to deserialize xml messages into objects.

In doing some memory analysis I've found the following that doesn't make any
sense to me so I was hoping someone could shed some light on the issue.

Simple Object contains a string, int and bool, as well as another complex
object that also contains a string, int and bool.

If I use code to loop through and create identical objects and add them to a
Hashtable, I see each object taking 430K

If I put a method within that object that 'clones' it and returns a new
object, the new object is only 234K.

Changing the scenario to using the XmlSerializer to deserialize the object
from a string, my memory size of a single object grows to 1016K.

If I call my 'clone' method on this object the size is reduced to 488K.

In my simple mind all of these instances are identical, however there are 4
different sizes which cover a large range.

I'm thinking of creating a CodeGeneration application that could create my
'clone' methods for me, but wanted to make sure that what I was seeing had
some logical justification behind it.

Also, does this behavior change in .NET 2.0 or 3.0? Is there any way for me
to get my 'deserialized' version of the object to be down to the 234K size?

Thanks

Reply With Quote
  #7  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:30 PM



Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

It's a console app that creates all this and then waits. I show heap with
CLR Profiler and that is where I'm getting the measurements. Do you have a
better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

"Dave Hiniker - MSFT" wrote:

Quote:
Could you clarify the contents of the object (what makes it hundreds of K)?
And how are you measuring the memory overhead in each case? Thanks!

"Michael Cline" wrote:

Framework 1.1
Our environment requires us to deserialize xml messages into objects.

In doing some memory analysis I've found the following that doesn't make any
sense to me so I was hoping someone could shed some light on the issue.

Simple Object contains a string, int and bool, as well as another complex
object that also contains a string, int and bool.

If I use code to loop through and create identical objects and add them to a
Hashtable, I see each object taking 430K

If I put a method within that object that 'clones' it and returns a new
object, the new object is only 234K.

Changing the scenario to using the XmlSerializer to deserialize the object
from a string, my memory size of a single object grows to 1016K.

If I call my 'clone' method on this object the size is reduced to 488K.

In my simple mind all of these instances are identical, however there are 4
different sizes which cover a large range.

I'm thinking of creating a CodeGeneration application that could create my
'clone' methods for me, but wanted to make sure that what I was seeing had
some logical justification behind it.

Also, does this behavior change in .NET 2.0 or 3.0? Is there any way for me
to get my 'deserialized' version of the object to be down to the 234K size?

Thanks

Reply With Quote
  #8  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-12-2006 , 06:30 PM



Dave,
Sorry, I realized after posting that I had been too close to the problem
and my description wasn't completely accurate.

I am creating an object over and over again and adding the new copy to a
hashtable.

In one scenario I would manually set the properties of the object, in
another scenario I would have serialized a copy of the object and I would be
reading it in instead of new ComplexObj(). In both cases I have a 'deep
clone' written which creates a 'new' object of the main type and then of the
one complex type that it contains, populates this new object with the values
from itself and then returns that.

I'm measuring the size with the CLR Profiler. It gives me an accurate count
of objects, and a memory size of these objects. It doesn't make sense that
the size of these four 'versions' of exactly the same thing display 4
different sizes.

The tester is a console app that creates all this and then waits. I show
heap with CLR Profiler and that is where I'm getting the measurements. Do
you have a better mechanism for seeing .NET memory usage?
Any assistance you can offer would be appreciated.
Thanks,

Reply With Quote
  #9  
Old   
Michael Cline
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 10-13-2006 , 07:36 PM



Looks like on XP SP2 I only come up with 2 sets of memory sizes, one for the
manually constructed object and one for those generated via the
XmlSerializer. The 2.0 Framework seems to make the analyzation completely
different.

I e-mailed you the code and results, following is the code.

using System;
using System.IO;
using System.Collections;
using System.Xml.Serialization;

namespace Memory
{
public class Anomoly
{
public static void Main( )
{
//Test1();
Test2();
//Test3();
//Test4();
}

public static void Test1( )
{
Hashtable storage = new Hashtable();
for( int i = 0; i < 10000; i++ )
{
ComplexObject tempVehicle = BuildObject();
storage.Add( i, tempVehicle );
}
GC.Collect();
Console.WriteLine( "Storage has been populated" );
Console.Read();
}

public static void Test2( )
{
Hashtable storage = new Hashtable();
for( int i = 0; i < 10000; i++ )
{
ComplexObject tempVehicle = BuildObject();
storage.Add( i, tempVehicle.DeepClone() );
}
GC.Collect();
Console.WriteLine( "Storage has been populated with manually created
clones" );
Console.Read();
}

public static void Test3( )
{
StreamWriter serializedWriter = new StreamWriter("C:\\Object.xml");
XmlSerializer serializer = new XmlSerializer( typeof(ComplexObject) );
ComplexObject objectToSerialize = BuildObject();
serializer.Serialize( serializedWriter, objectToSerialize );
serializedWriter.Close();

StreamReader serializedReader = new StreamReader( "C:\\Object.xml" );
string serializedString = serializedReader.ReadToEnd();
serializedReader.Close();
Hashtable storage = new Hashtable();
for( int i = 0; i < 10000; i++ )
{
StringReader objectReader = new StringReader( serializedString );
ComplexObject tempVehicle = serializer.Deserialize( objectReader )
as ComplexObject;
storage.Add( i, tempVehicle );
objectReader.Close();
}
GC.Collect();
Console.WriteLine( "Storage has been populated with XmlDeserialized
Objects" );
Console.Read();
}

public static void Test4( )
{
StreamWriter serializedWriter = new StreamWriter("C:\\Object.xml");
XmlSerializer serializer = new XmlSerializer( typeof(ComplexObject) );
ComplexObject objectToSerialize = BuildObject();
serializer.Serialize( serializedWriter, objectToSerialize );
serializedWriter.Close();

StreamReader serializedReader = new StreamReader( "C:\\Object.xml" );
string serializedString = serializedReader.ReadToEnd();
serializedReader.Close();
Hashtable storage = new Hashtable();
for( int i = 0; i < 10000; i++ )
{
StringReader objectReader = new StringReader( serializedString );
ComplexObject tempVehicle = serializer.Deserialize( objectReader )
as ComplexObject;
storage.Add( i, tempVehicle.DeepClone() );
objectReader.Close();
}
GC.Collect();
Console.WriteLine( "Storage has been populated with XmlDeserialized
Objects" );
Console.Read();
}


private static ComplexObject BuildObject()
{
SimpleObject tempPerson = new SimpleObject();
tempPerson.Irritable = true;
tempPerson.Age = 45;
tempPerson.Name = "John";

ComplexObject rc = new ComplexObject();
rc.Wheels = 4;
rc.Model = "Ford";
rc.Available = true;
rc.Person = tempPerson;
return rc;
}
}
public class SimpleObject
{
public SimpleObject(){}

private int age = 0;
private string name = string.Empty;
private bool irritable = false;

public int Age
{
get{return age;}
set{age = value;}
}

public string Name
{
get{return name;}
set{name = value;}
}

public bool Irritable
{
get{return irritable;}
set{irritable = value;}
}
}

public class ComplexObject
{
public ComplexObject(){}

private SimpleObject person = null;
private int wheels = 0;
private string model = string.Empty;
private bool available = false;

public SimpleObject Person
{
get{return person;}
set{person = value;}
}

public int Wheels
{
get{return wheels;}
set{wheels = value;}
}

public string Model
{
get{return model;}
set{model = value;}
}

public bool Available
{
get{return available;}
set{available = value;}
}

public ComplexObject DeepClone()
{
ComplexObject rc = new ComplexObject();
rc.Wheels = wheels;
rc.Model = model;
rc.Available = available;

if( null != person )
{
SimpleObject tempPerson = new SimpleObject();
tempPerson.Irritable = Person.Irritable;
tempPerson.Age = Person.Age;
tempPerson.Name = Person.Name;
rc.Person = tempPerson;
}
return rc;
}
}
}


Reply With Quote
  #10  
Old   
Dave Black
 
Posts: n/a

Default RE: XmlSerialization Object Memory Usage - 12-27-2007 , 11:33 AM



Hi Michael,

I'm not sure if you're referring to a memory leak. If so, there is a known
issue with XmlSerializer which affects .NET 1.x and 2.0. If the types of
objects being serialized are known at compile-time, best practice is to
declare a static instance of the XmlSerializer for that type.

You can read more about the XmlSerializer memory leak at:
http://blogs.msdn.com/tess/archive/2...15/532804.aspx and
http://support.microsoft.com/kb/886385/en-us

Note that there are also known memory leaks with script blocks in XSL
Transforms:
http://support.microsoft.com/?id=316775

HTH
--
2b || !2b


"Michael Cline" wrote:

Quote:
Framework 1.1
Our environment requires us to deserialize xml messages into objects.

In doing some memory analysis I've found the following that doesn't make any
sense to me so I was hoping someone could shed some light on the issue.

Simple Object contains a string, int and bool, as well as another complex
object that also contains a string, int and bool.

If I use code to loop through and create identical objects and add them to a
Hashtable, I see each object taking 430K

If I put a method within that object that 'clones' it and returns a new
object, the new object is only 234K.

Changing the scenario to using the XmlSerializer to deserialize the object
from a string, my memory size of a single object grows to 1016K.

If I call my 'clone' method on this object the size is reduced to 488K.

In my simple mind all of these instances are identical, however there are 4
different sizes which cover a large range.

I'm thinking of creating a CodeGeneration application that could create my
'clone' methods for me, but wanted to make sure that what I was seeing had
some logical justification behind it.

Also, does this behavior change in .NET 2.0 or 3.0? Is there any way for me
to get my 'deserialized' version of the object to be down to the 234K size?

Thanks

Reply With Quote
Reply




Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.4
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.