Serviced Component : Architectural revival

We have ASP.net application which uses crystal report as a report generation
tool and SQL server Database server.
We are using Microisoft Entlib Security Blcok and Dataacess block.
We are facing a huge performance degeration problem in the application
mainly in reporting section.
We audited the scenario in our test labs and found out that it took 17 secs
(avg.) to get the data from the DB (for a particular compute insentive
report) and almost 2 1/2 mins to compile the data into a pdf stream (of
almost 150 pgs report) through Crystal report. Because of this report IIS
worker processor is consuming huge amount of memory and when reports get
generated whole application get slow.
We increasased the connection timeout property, it helped generating the
report but still, since data is pilling up, performance is degreading a lot.
We have formalized following recommendations for optimizing it:
1. DB optmization
2. Placement of Dataacess layer to other machine.

For placement of Dataacess layer to other machine we can make a. Datacess
block remotable, but then it means that we have to change huge amount of code
b. We can make service components, which will give us additional facility of
1. Object Pooling
2. JIT
3. Asynchronous call

Keeping all this scenario in mind i need following suggestions, since the
whole decission is on make or break position.
1. Should we use service components or remoting ?
2. Can serviced components help us out under such circumstances?
3. Can we use batch processing of reports using serviced components?

Hello Furqan,

FF> We have ASP.net application which uses crystal report as a report
FF> generation
FF> tool and SQL server Database server.
FF> We are using Microisoft Entlib Security Blcok and Dataacess block.
FF> We are facing a huge performance degeration problem in the
FF> application
FF> mainly in reporting section.
FF> We audited the scenario in our test labs and found out that it took
FF> 17 secs
FF> (avg.) to get the data from the DB (for a particular compute
FF> insentive
FF> report)

Did you compare this querying time with one by using SQL Management studio?
Are these 2 queries relevant by time?


FF> and almost 2 1/2 mins to compile the data into a pdf stream
FF> (of
FF> almost 150 pgs report) through Crystal report.

Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?


FF> Because of this report IIS
FF> worker processor is consuming huge amount of memory and when reports
FF> get
FF> generated whole application get slow.

Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.

I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing


FF> For placement of Dataacess layer to other machine we can make a.
FF> Datacess
FF> block remotable, but then it means that we have to change huge
FF> amount of code
FF> b. We can make service components, which will give us additional
FF> facility of
FF> 1. Object Pooling
FF> 2. JIT
FF> 3. Asynchronous call
FF> Keeping all this scenario in mind i need following suggestions,
FF> since the
FF> whole decission is on make or break position.
FF> 1. Should we use service components or remoting ?

Service component is more productive

FF> 2. Can serviced components help us out under such circumstances?

hope it could. You need to move intensive processing to other layer, which
is more appropriate for this.


FF> 3. Can we use batch processing of reports using serviced components?

sure


Looking forwart to see you answers

---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

Thanks Michael for your reply
Q. Did you compare this querying time with one by using SQL Management
studio?
Are these 2 queries relevant by time?
A. Nopes. We captured the time by adding intrunmentaions in Data Access Layer

Q. Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?
A. Nopes. Our client have Crystal Report license and they export it in Excel
as well as PDF. PDF is the default format of report generation

Q. Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?
A. It's in IIS worker process . Report is generated by creating a PDF
stream through Crytal Report Engine. We pass that stream to the Response
object.

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.
I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing
Comment: We realized that there's problem in the database deisgn as well but
then there's lots of rows. A ingle report is like of almost 40 - 45 pages.

Service component is more productive
Comment: We are looking for a solution haveing less Time To Market, since
its a Core Business Application at the same time, in the near future, we are
looking forward of buying a seperate server for hsoting application's Data
Acess Block.
Under such circumstances do you think that COM+ apps will suite the need?
I dont know how to call COM+ apps remotely. Do have any article or book
recommendation ?


Thanks again.. looking forward for your reply....







--
Mohammad Furqan Baqai
MCAD


"Michael Nemtsev" wrote:

Hello Furqan,

FF> We have ASP.net application which uses crystal report as a report
FF> generation
FF> tool and SQL server Database server.
FF> We are using Microisoft Entlib Security Blcok and Dataacess block.
FF> We are facing a huge performance degeration problem in the
FF> application
FF> mainly in reporting section.
FF> We audited the scenario in our test labs and found out that it took
FF> 17 secs
FF> (avg.) to get the data from the DB (for a particular compute
FF> insentive
FF> report)

Did you compare this querying time with one by using SQL Management studio?
Are these 2 queries relevant by time?


FF> and almost 2 1/2 mins to compile the data into a pdf stream
FF> (of
FF> almost 150 pgs report) through Crystal report.

Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?


FF> Because of this report IIS
FF> worker processor is consuming huge amount of memory and when reports
FF> get
FF> generated whole application get slow.

Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.

I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing


FF> For placement of Dataacess layer to other machine we can make a.
FF> Datacess
FF> block remotable, but then it means that we have to change huge
FF> amount of code
FF> b. We can make service components, which will give us additional
FF> facility of
FF> 1. Object Pooling
FF> 2. JIT
FF> 3. Asynchronous call
FF> Keeping all this scenario in mind i need following suggestions,
FF> since the
FF> whole decission is on make or break position.
FF> 1. Should we use service components or remoting ?

Service component is more productive

FF> 2. Can serviced components help us out under such circumstances?

hope it could. You need to move intensive processing to other layer, which
is more appropriate for this.


FF> 3. Can we use batch processing of reports using serviced components?

sure


Looking forwart to see you answers

---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

Furqan,

Hmm... may be you can tune your database access and PDF generation to
be a lot faster, but there is a chance that there is nothing really
wrong with your implementation, it just simply takes that long to
process your work. If that is the case, you may want to reconsider
your workflow... rather than doing all these in ASPX/IIS, may be do
your processing in another process and send out an email to the user
once it is done?

On COM+, since it is actually implemented in unmanaged code, you would
have some additional "leaked abstractions" that you have to take care
for when throwing custom exceptions from your serviced components to
your client applications... see
http://msdn.microsoft.com/msdnmag/is...ceptionsinCOM/

HTH,
Alex

Furqan Baqai wrote:

Thanks Michael for your reply
Q. Did you compare this querying time with one by using SQL Management
studio?
Are these 2 queries relevant by time?
A. Nopes. We captured the time by adding intrunmentaions in Data Access Layer

Q. Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?
A. Nopes. Our client have Crystal Report license and they export it in Excel
as well as PDF. PDF is the default format of report generation

Q. Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?
A. It's in IIS worker process . Report is generated by creating a PDF
stream through Crytal Report Engine. We pass that stream to the Response
object.

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.
I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing
Comment: We realized that there's problem in the database deisgn as well but
then there's lots of rows. A ingle report is like of almost 40 - 45 pages.

Service component is more productive
Comment: We are looking for a solution haveing less Time To Market, since
its a Core Business Application at the same time, in the near future, we are
looking forward of buying a seperate server for hsoting application's Data
Acess Block.
Under such circumstances do you think that COM+ apps will suite the need?
I dont know how to call COM+ apps remotely. Do have any article or book
recommendation ?


Thanks again.. looking forward for your reply....







--
Mohammad Furqan Baqai
MCAD


"Michael Nemtsev" wrote:

Hello Furqan,

FF> We have ASP.net application which uses crystal report as a report
FF> generation
FF> tool and SQL server Database server.
FF> We are using Microisoft Entlib Security Blcok and Dataacess block.
FF> We are facing a huge performance degeration problem in the
FF> application
FF> mainly in reporting section.
FF> We audited the scenario in our test labs and found out that it took
FF> 17 secs
FF> (avg.) to get the data from the DB (for a particular compute
FF> insentive
FF> report)

Did you compare this querying time with one by using SQL Management studio?
Are these 2 queries relevant by time?


FF> and almost 2 1/2 mins to compile the data into a pdf stream
FF> (of
FF> almost 150 pgs report) through Crystal report.

Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?


FF> Because of this report IIS
FF> worker processor is consuming huge amount of memory and when reports
FF> get
FF> generated whole application get slow.

Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.

I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing


FF> For placement of Dataacess layer to other machine we can make a.
FF> Datacess
FF> block remotable, but then it means that we have to change huge
FF> amount of code
FF> b. We can make service components, which will give us additional
FF> facility of
FF> 1. Object Pooling
FF> 2. JIT
FF> 3. Asynchronous call
FF> Keeping all this scenario in mind i need following suggestions,
FF> since the
FF> whole decission is on make or break position.
FF> 1. Should we use service components or remoting ?

Service component is more productive

FF> 2. Can serviced components help us out under such circumstances?

hope it could. You need to move intensive processing to other layer, which
is more appropriate for this.


FF> 3. Can we use batch processing of reports using serviced components?

sure


Looking forwart to see you answers

---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

Thanks Li
As far as changing the process of report generation is concerned, we are on
our way to implement it.
We have drafted the implementation plan and according to it we are using
following solution:
1. Making Dataacess Layer as serviced component
2. For reports which are taking lot of time, we are implementing
asynchronous remoting, which will generat PDF and instead of sending the
stream, it will write the file in the HDD. Status will be updated in the
database. User can view the status through a interface.

We are usig COM+ since we can have Object Pooling and JIT feature which will
help boosting the application performance.

The only problem i have is that should i use COM+ objects or not? Since it
will complicate the dsign of the application.

Mohammad Furqan Baqai
MCAD


"Alex Li" wrote:

Furqan,

Hmm... may be you can tune your database access and PDF generation to
be a lot faster, but there is a chance that there is nothing really
wrong with your implementation, it just simply takes that long to
process your work. If that is the case, you may want to reconsider
your workflow... rather than doing all these in ASPX/IIS, may be do
your processing in another process and send out an email to the user
once it is done?

On COM+, since it is actually implemented in unmanaged code, you would
have some additional "leaked abstractions" that you have to take care
for when throwing custom exceptions from your serviced components to
your client applications... see
http://msdn.microsoft.com/msdnmag/is...ceptionsinCOM/

HTH,
Alex

Furqan Baqai wrote:
Thanks Michael for your reply
Q. Did you compare this querying time with one by using SQL Management
studio?
Are these 2 queries relevant by time?
A. Nopes. We captured the time by adding intrunmentaions in Data Access Layer

Q. Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?
A. Nopes. Our client have Crystal Report license and they export it in Excel
as well as PDF. PDF is the default format of report generation

Q. Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?
A. It's in IIS worker process . Report is generated by creating a PDF
stream through Crytal Report Engine. We pass that stream to the Response
object.

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.
I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing
Comment: We realized that there's problem in the database deisgn as well but
then there's lots of rows. A ingle report is like of almost 40 - 45 pages.

Service component is more productive
Comment: We are looking for a solution haveing less Time To Market, since
its a Core Business Application at the same time, in the near future, we are
looking forward of buying a seperate server for hsoting application's Data
Acess Block.
Under such circumstances do you think that COM+ apps will suite the need?
I dont know how to call COM+ apps remotely. Do have any article or book
recommendation ?


Thanks again.. looking forward for your reply....







--
Mohammad Furqan Baqai
MCAD


"Michael Nemtsev" wrote:

Hello Furqan,

FF> We have ASP.net application which uses crystal report as a report
FF> generation
FF> tool and SQL server Database server.
FF> We are using Microisoft Entlib Security Blcok and Dataacess block.
FF> We are facing a huge performance degeration problem in the
FF> application
FF> mainly in reporting section.
FF> We audited the scenario in our test labs and found out that it took
FF> 17 secs
FF> (avg.) to get the data from the DB (for a particular compute
FF> insentive
FF> report)

Did you compare this querying time with one by using SQL Management studio?
Are these 2 queries relevant by time?


FF> and almost 2 1/2 mins to compile the data into a pdf stream
FF> (of
FF> almost 150 pgs report) through Crystal report.

Did you tried to wrote test of generating pdf by means of any other tools,
like http://csharp-source.net/open-source/pdf-libraries?


FF> Because of this report IIS
FF> worker processor is consuming huge amount of memory and when reports
FF> get
FF> generated whole application get slow.

Where this pdf convertion performs? I hope not in the IIS work process
Could you describe process of converting more wide?

FF> We increasased the connection timeout property, it helped generating
FF> the report but still, since data is pilling up, performance is
FF> degreading a lot.

FF> 1. DB optmization
FF> 2. Placement of Dataacess layer to other machine.

I'm not sure if this helps. The problem not in the numbers of request, but
in the data processing


FF> For placement of Dataacess layer to other machine we can make a.
FF> Datacess
FF> block remotable, but then it means that we have to change huge
FF> amount of code
FF> b. We can make service components, which will give us additional
FF> facility of
FF> 1. Object Pooling
FF> 2. JIT
FF> 3. Asynchronous call
FF> Keeping all this scenario in mind i need following suggestions,
FF> since the
FF> whole decission is on make or break position.
FF> 1. Should we use service components or remoting ?

Service component is more productive

FF> 2. Can serviced components help us out under such circumstances?

hope it could. You need to move intensive processing to other layer, which
is more appropriate for this.


FF> 3. Can we use batch processing of reports using serviced components?

sure


Looking forwart to see you answers

---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

Hello Furqan,

FB> As far as changing the process of report generation is concerned, we
FB> are on
FB> our way to implement it.
FB> We have drafted the implementation plan and according to it we are
FB> using
FB> following solution:
FB> 1. Making Dataacess Layer as serviced component
FB> 2. For reports which are taking lot of time, we are implementing
FB> asynchronous remoting, which will generat PDF and instead of sending
FB> the
FB> stream, it will write the file in the HDD. Status will be updated in
FB> the
FB> database. User can view the status through a interface.
FB> We are usig COM+ since we can have Object Pooling and JIT feature
FB> which will help boosting the application performance.

Fine. it could really help you

FB> The only problem i have is that should i use COM+ objects or not?
FB> Since it will complicate the dsign of the application.

And what alternative? Everything that you need is to move you process consuming
part from the IIS.
COM+ is a good way for this. It's no so hard as you thing. COM+ under .net
is Enterprise Services.
See MSDN for more info

---
WBR,
Michael Nemtsev :: blog: http://spaces.msn.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

Furqan Baqai wrote:

We are usig COM+ since we can have Object Pooling and JIT feature which will
help boosting the application performance.

The only problem i have is that should i use COM+ objects or not? Since it
will complicate the dsign of the application.

Using COM+ does complicate the design and maintenance; however COM+
allows you to control how many objects you can have on each server (max
and min)... that is handy for resources which can have a limit (ie, MQ
connections, SQL connections) or you server is also doing something
else and you don't want your PDF generation process to use up all the
available resources ... And other factor would be the startup/teardown
costs of your objects... if they are expensive compare to your other
actions, then it would make sense to use COM+ to minimize that... It
is really hard to say if you really need to use COM+ or not; the only
way to find out is basically through prototyping.

HTH,
Alex

My 2c here....

I would vote for COM+ as there are lot of benefits that you won't find
in remoting. The deployment maybe slightly more complicated but you get
all these things that were mentioned...

I personally believe that the provided services plus the flexibility in
deployment is a great reason to use COM+ (at least they justify the
additional cost of development and deployment)

Regards,
Tasos

Furqan,

We have a customer that had a similar problem. They build PDF files from
their web application. In the initial design of their product the PDF
construction was happening on the web server which was slowing things down
for all their users. They solved their problem by moving the PDF file
generation off the web server and onto a compute grid. You can read their
case study here: http://www.digipede.net/products/case_pacevents.html and
if you want to try using the Digipede Network, the Digipede Network Developer
Edition is free and you can get that here:
http://www.digipede.net/products/dev-edition.html.

Good luck,

Kim Greenlee

--
digipede - Many legs make light work.
Grid computing for the real world.
http://www.digipede.net
http://krgreenlee.blogspot.net

Just as a note about Crystal Reports. The report generation engine for
Crystal is limited to 2 instances per server unless you shell out the cash
for the Enterprise Reporting Server (or whatever they are calling it these
days). I ran into this with a client and we set up some beefy servers to
generate reports out of process. While this solved the problem on the
application server the client was unhappy to learn that Crystal's objects
were forcibly limiting the instance count to 2.

I can provide more info if you need it but we wound up having to farm out
our report generation requests across several machines in order to handle
our report generation requirements without shelling out the money for the
server version of crystal.

-DS


"Furqan Baqai" <Furqan Baqai (AT) discussions (DOT) microsoft.com> wrote

We have ASP.net application which uses crystal report as a report
generation
tool and SQL server Database server.
We are using Microisoft Entlib Security Blcok and Dataacess block.
We are facing a huge performance degeration problem in the application
mainly in reporting section.
We audited the scenario in our test labs and found out that it took 17
secs
(avg.) to get the data from the DB (for a particular compute insentive
report) and almost 2 1/2 mins to compile the data into a pdf stream (of
almost 150 pgs report) through Crystal report. Because of this report IIS
worker processor is consuming huge amount of memory and when reports get
generated whole application get slow.
We increasased the connection timeout property, it helped generating the
report but still, since data is pilling up, performance is degreading a
lot.
We have formalized following recommendations for optimizing it:
1. DB optmization
2. Placement of Dataacess layer to other machine.

For placement of Dataacess layer to other machine we can make a. Datacess
block remotable, but then it means that we have to change huge amount of
code
b. We can make service components, which will give us additional facility
of
1. Object Pooling
2. JIT
3. Asynchronous call

Keeping all this scenario in mind i need following suggestions, since the
whole decission is on make or break position.
1. Should we use service components or remoting ?
2. Can serviced components help us out under such circumstances?
3. Can we use batch processing of reports using serviced components?