Re: AppDomain and Memory Issues

I'm experiencing this same thing as well. Any ideas?

Baddabing wrote:

JM,

I've actually been going nuts over this AppDomain issue and am about
to give
up. However, I believe I was wrong in stating that this is only an
issue
with the VB VSA engine.

My experiments lead me to now believe that continually instantiating
(different each time) assemblies into a secondary AppDomain which is
then
unloaded, leads to a substantial increase in system memory use -
recoverable
only at end of process. This is the case even if interaction between
the
default and secondary AppDomain is limited to invoking of methods
through an
interface returned through a factory created instance of the assembly
in the
second AppDomain - i.e. the default AppDomain knows nothing about the
remote
assembly other than through that interface.

This doesn't really impact on generating a small number of dynamic
assemblies for things like plugins, but sure does for scripting where
one
might want to generate and load hundreds of assemblies.

Would be really grateful if someone could show me I'm wrong?

Your project sounds interesting - is it web based?

Best regards,
Baddabing



"JM Servera" <soylandream (AT) yahoo (DOT) es> wrote in message
news:#YyPAjtYCHA.2260 (AT) tkmsftngp09 (DOT) ..
Hi Baddabing!
Thanks for your explanation. I was in the belief that the
Garbage
Collector will someday free the memory.
As you, I'm very disappointed with having to use a second
AppDomain to
allow my app unloading the script.
I'm developing a service that will run scheduled scripts and I
need to
unload them every time that they have run, so, if the service has
this
memory leaks then I can't put it in production... and this will not
make
my
boss very happy.
So, I'm also very interested in finding anyone who found the
way to
"correctly" unload the VB engine. (Actually, not only interested, I
do
need
it).
Thanks again,
JM

"Baddabing" <reply (AT) togroup (DOT) com> escribió en el mensaje
news:O8JB6#PYCHA.2452 (AT) tkmsftngp11 (DOT) ..
Hi JM,

This isn't a managed memory issue.

Simply instantiating the VB Engine in a second AppDomain, and
then
successfully unloading that AppDomain, leaves allocated system
memory.
As
mentioned this is not visibly the case for the JScript engine or
for an
arbitrary assembly generated through the Reflection.Emit
namespace into
a
non-primary domain.

On my system, creating an instance of the VB engine into a second
AppDomain
and unloading the AppDomain grows system memory usage by (very)
approx.
4.5mb every 100 instantiations.

All I can think of, is that this is a domain-neutral assembly
issue and
the
VSA VB code (domain-neutral assemblies statics and CLR data
structures
are
freed on an AppDomain unload but not code) is huge or something.
Seems
unlikely to me, but it could be tested by implementing ones own
CLR host
with CorBindToRuntimeEx specifying no domain-neutral assemblies
(other
than
mscorlib.dll).

As per my previous message I think that having to use a second
AppDomain
(for a compile many/run few scenario) negates the real advantages
of
Script
for the .Net Framework (live objects and events). Furthermore,
I'd
hazard
a
guess that the script team have let out a few expletives about
the CLR's
inability to unload an assembly. If one could - scripting would
be
soooo
schweeet.

But would be useful to know if anyone has implemented a VB engine
AppDomain
recycling scheme where system memory doesn't grow as I describe?
Anyone
have any other thoughts or a solution?

Regards,
Baddabing

"JM Servera" <soylandream (AT) yahoo (DOT) es> wrote in message
news:#tIchwLYCHA.3736 (AT) tkmsftngp08 (DOT) ..
I'm testing this application with the allocation profiler (




http://www.gotdotnet.com/userarea/fi...locationProfil
er.zip ) and seems to work ok, think that GC.GetTotalMemory
gives you
just
an estimation of the allocated heap.
"Baddabing" <reply (AT) togroup (DOT) com> escribió en el mensaje
news:eaUfvkJYCHA.1732 (AT) tkmsftngp10 (DOT) ..
Trent,

Exactly - that's the idea. One point though, is that
typically the
interface and class factory would be created in their own
DLL.

The really bad news is that you will find that memory
continues to
be
consumed when instantiating (thats all it takes) the VB
engine in a
second
AppDomain, even after that second AppDomain is unloaded.
This is
not
the
case for the JScript engine.

( MS - I would really like to know the cause of this VB
engine
unload
memory
issue - is it related to there being no mechanism to fully
unload a
domain-neutral assembly?)

This flaw I think prevents the use of the VB engine in this
scenario.
Microsoft, I believe, have stated that it was never intended
the use
of
a
second AppDomain solution to solve the assembly unload
dilemma. The
whole
point of their solution is access to live objects and events
which
is
negated by having to use a second AppDomain.

The necessity to use a second AppDomain in a compile many
times
scenario,
and then pay the price in regards to live objects and events,
means
that
Script for the Dotnet framework remains suitable only for a
compile
once/run
many times scenario IMHO.

Regards,
Baddabing

"Trent" <nojunk (AT) mail (DOT) com> wrote in message
news:#WVWVFDYCHA.444 (AT) tkmsftngp12 (DOT) ..
I have attached a working copy of the of AppDomain
recycling
scheme
that
I
put together from the articles. hopefully this is the
right
way???

Trent


"Trent" <nojunk (AT) mail (DOT) com> wrote in message
news:OIYfvcAYCHA.2452 (AT) tkmsftngp09 (DOT) ..
Hello Baddabing

Thanks for the ideas. I have not been successfully able
to
implement
them
yet, but I'm still trying. The link to the article you
mention;
(watch
for
word wrap)








http://msdn.microsoft.com/library/de...us/dncscol/htm
l/csharp05162002.asp

another usefully article can be found at Codeproject

http://www.codeproject.com/csharp/livecodedotnet.asp

If anybody has been successful in implementing AppDomain
recycling
schem
e
using the VSA engine please let me know

Thanks

Trent

"Baddabing" <reply (AT) togroup (DOT) com> wrote in message
news:#H9Ppw6XCHA.2652 (AT) tkmsftngp10 (DOT) ..
Trent,

I haven't tested it but I suspect your problem is
related to
maintaining
isolation between your AppDomains. The way you have
written
your
code, the AppDomainTesting.TestEngine object is still
being
attached
to
your
primary AppDomain because of the
CreateInstanceAndUnwrap call
(and
cast)
which is returning a (proxy) TestEngine object.
Although it
is
a
proxy,
the
primary AppDomain still attaches to the assembly as
metadata
is
required.

To solve this, an indirection solution (yuck) where the
primary
AppDomain
has NO knowledge of the assembly is required (class
factory
and
invoke
by
interface?).

Eric Gunnerson has recently written an article on
dynamic
assemblies
on
MSDN
which may help (sorry can't find the reference).

For this reason, I can't see how the AppDomain
recycling
scheme
supplied
by
MS in the thread of 15 July 2002 could work. Has
anyone
actually
managed to implement it successfully - how?

Regards,
Baddabing

"Trent" <nojunk (AT) mail (DOT) com> wrote in message
news:u3e06BoWCHA.436 (AT) tkmsftngp10 (DOT) ..
I have been working on loading and unload the VSA
Engine
into
AppDomain
to
reduce the amount of memory usage in my application.
I have
built
a
test
environment that will create and unload a VSA Engine
into a
AppDomain.
While this reduces the amount of memory used compared
to
simple
creating
and
destroying the VAS Engine in the one domain, I still
have an
increase
in
memory.

Have I set up the configuration for the domain
incorrectly
or
is
this
the
wrong way.

I have attached the project and the code it anybody
would
like
to
test
it.
You will notice an increase in memory if the engine
is
complied.

Test (With a for loop of 100)
Without Compile Memory Usage = 15,000 k
With compile Memory usage = 30,000 K

While these memory values are low, it cause problems
in the
production
environment when the number of engines and complexity
of the
script
increase.

Thanks

using System;
using System.Reflection;
using System.Windows.Forms;
using System.Collections;
namespace AppDomainTesting
{
/// <summary
/// Summary description for Class1.
/// </summary
class Class1
{
/// <summary
/// The main entry point for the application.
/// </summary
[STAThread]
static void Main(string[] args)
{
for(int i=0; i<100; i++)
{
AppDomain appDomain = null;
AppDomainSetup setup = new AppDomainSetup();
AppDomainSetup appDomainSetUp = new
AppDomainSetup();

Console.WriteLine("Insatance = "+i.ToString());
try
{

appDomain =









AppDomain.CreateDomain(System.Guid.NewGuid().ToStr ing(),null,appDomainSetUp)
;

string assemblyName = Assembly.GetAssembly(
typeof(AppDomainTesting.ScriptEngine) ).FullName;

AppDomainTesting.TestEngine test =









(AppDomainTesting.TestEngine)appDomain.CreateInsta nceAndUnwrap(assemblyName,
"AppDomainTesting.TestEngine");
//comment out the next line and memory usage
will be
lower
test.Compile();
test.Run();

}
catch(System.Exception err)
{
string error = err.ToString();
System.Windows.Forms.MessageBox.Show(error);
}
finally
{
if(appDomain != null)
AppDomain.Unload(appDomain);
}

}

Console.WriteLine("Total Memory =" +
GC.GetTotalMemory(true).ToString());
Console.WriteLine("Press Enter To End");
if(Console.Read()>0)
return;
}
}
public class TestEngine :
System.ComponentModel.Component
{
private ScriptEngine engine;
public TestEngine()
{
engine = new ScriptEngine();
}
public void Compile()
{
engine.Compile();
}
public void Run()
{

if(engine.IsCompiled)
{
engine.Run();

string strNamespace = engine.RootNamespace;
string FullName = strNamespace +".script";

// Get the method from the type
Type ModType =
engine.Assembly.GetType(FullName,true,true);
System.Reflection.MethodInfo method =
ModType.GetMethod("main");
if (method != null)
method.Invoke(null,null);
}
}
}
public class ScriptEngine :
Microsoft.VisualBasic.Vsa.VsaEngine
{
public ScriptEngine()
{
this.RootMoniker = "MemoryTesting://script";
this.Site = new VsaBaseSite();
this.InitNew();
this.RootNamespace = "scriptcode";
this.Name = "scriptcode";
Microsoft.Vsa.IVsaReferenceItem systemRef=









(Microsoft.Vsa.IVsaReferenceItem)this.Items.Create Item("system.dll",Microsof
t.Vsa.VsaItemType.Reference,
Microsoft.Vsa.VsaItemFlag.None);
systemRef.AssemblyName = "System.dll";
Microsoft.Vsa.IVsaCodeItem code=









(Microsoft.Vsa.IVsaCodeItem)this.Items.CreateItem( "main",Microsoft.Vsa.VsaIt
emType.Code, Microsoft.Vsa.VsaItemFlag.Module);
code.SourceText= this.script;

}
private string script = "Option Strict Off\n"+
"Option Explicit Off\n"+
"Imports System\n"+
"Imports System.Windows.Forms\n"+
"Public Module script\n"+
"Sub main\n"+
//"MessageBox.Show(\"Testing\")\n"+
"Console.WriteLine(\"Testing \" &
DateTime.Now.ToLongTimeString())\n"+
//"MessageBox.Show(\"Testing\")\n"+
"End Sub\n"+
"End Module\n";
}
public class VsaBaseSite :Microsoft.Vsa.IVsaSite
{


#region Implementation of IVsaSite
public object GetEventSourceInstance(string
itemName,
string
eventSourceName)
{
return null;
}
public object GetGlobalInstance(string name)
{
return null;
}
public void Notify(string notify, object info)
{

}
public bool OnCompilerError(Microsoft.Vsa.IVsaError
error)
{
Console.WriteLine(error.ToString());
return true;
}
public void GetCompiledState(out byte[] pe, out
byte[]
debugInfo)
{
pe = null;
debugInfo = null;

}

#endregion
}
}