File Storage Question

In a web farm environment, how to store the user uploading files for future
access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file system?
or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the file
structure for better access
from file system?

Depends on what you want to do with the files and what they represent. Since
you haven't said, ASP.NET uploads files into server memory, buffers it and
then writes it to disk in the simplest case. Where you choose to store it
depends on how frequently you want to access it and the responsiveness you
want from the application, among other things.

--
--
Regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
The O.W.C. Black Book, 2nd Edition
Exclusively on www.lulu.com/owc $19.99


"Redhairs" <redhair@u.s.a> wrote

In a web farm environment, how to store the user uploading files for
future access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file
system? or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the file
structure for better access
from file system?

Thanks for your reply.
Just like simple photo album web app for user to upload image files and
review them later.


"Alvin Bruney [ASP.NET MVP]" <www.lulu.com/owc> wrote

Depends on what you want to do with the files and what they represent.
Since you haven't said, ASP.NET uploads files into server memory, buffers
it and then writes it to disk in the simplest case. Where you choose to
store it depends on how frequently you want to access it and the
responsiveness you want from the application, among other things.

--
--
Regards,
Alvin Bruney [MVP ASP.NET]

[Shameless Author plug]
The O.W.C. Black Book, 2nd Edition
Exclusively on www.lulu.com/owc $19.99


"Redhairs" <redhair@u.s.a> wrote in message
news:#GHoU4jTIHA.5788 (AT) TK2MSFTNGP06 (DOT) phx.gbl...
In a web farm environment, how to store the user uploading files for
future access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file
system? or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the
file structure for better access
from file system?

In solutions that I've designed that have this as a requirement, I've
typically used Network Attached Storage.

In most cases, all the servers in the web farm are on the same gigabit
network segment, and the NAS (which is usually multi-homed) sits on that
same segment.

In most cases, storing files in the database will lead to long term
problems. As amazing as current databases are, they're still not file
systems...

--
Chris Mullins

"Redhairs" <redhair@u.s.a> wrote

In a web farm environment, how to store the user uploading files for
future access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file
system? or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the file
structure for better access
from file system?

Chris,

What type of long term problems?
Can you share please I store images in a database.

Thank You,

LVP


"Chris Mullins [MVP - C#]" <cmullins (AT) yahoo (DOT) com> wrote

In solutions that I've designed that have this as a requirement, I've
typically used Network Attached Storage.

In most cases, all the servers in the web farm are on the same gigabit
network segment, and the NAS (which is usually multi-homed) sits on that
same segment.

In most cases, storing files in the database will lead to long term
problems. As amazing as current databases are, they're still not file
systems...

--
Chris Mullins

"Redhairs" <redhair@u.s.a> wrote in message
news:%23GHoU4jTIHA.5788 (AT) TK2MSFTNGP06 (DOT) phx.gbl...
In a web farm environment, how to store the user uploading files for
future access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file
system? or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the
file structure for better access
from file system?

Any tips on file structure design based on performance and maintenance
concerns?
e.g. one folder for all image files? or many subfolders?

"Chris Mullins [MVP - C#]" <cmullins (AT) yahoo (DOT) com> wrote

In solutions that I've designed that have this as a requirement, I've
typically used Network Attached Storage.

In most cases, all the servers in the web farm are on the same gigabit
network segment, and the NAS (which is usually multi-homed) sits on that
same segment.

In most cases, storing files in the database will lead to long term
problems. As amazing as current databases are, they're still not file
systems...

--
Chris Mullins

"Redhairs" <redhair@u.s.a> wrote in message
news:%23GHoU4jTIHA.5788 (AT) TK2MSFTNGP06 (DOT) phx.gbl...
In a web farm environment, how to store the user uploading files for
future access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file
system? or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the
file structure for better access
from file system?

The biggest problems I've seen with storing files in a database is that it
grows without bound, and current databases don't seem to do very well with
really large data sets. In the large range (1TB -> 1PB) the file systems (in
my experience) deals with large sets more effectivly. The database is great
for storing "Which computers have that image on it?", but for the actual
image.... not so hot.

If you are adding files quickly enough, you'll need to be adding disk space
into your database every hours / days / weeks. Even with current large
drives, this is both expensive and time consuming. Using a DB, there's
really no good way to scatter the data across a large number of computers -
you could use FileGroup in SQL2K5 (Enterprise only), or something similar,
but these are not very good for dynamically adding data.

... if you're not adding drives, you're archiving files. This means a job
that runs, and moves images from the DB to a disk. You then need to design
the system to first check the DB, then check the disk. This makes things 2x
as complicated as just skipping the DB step and storing files on disk.

My preference at this point is:
- Store files on a NAS, or a set of NAS devices.
- Store the index (file names, metadata, location[s]) in the database.

I would love to hear of an out-of-the-box approach people have used for
this - especially in Oracle land (I'm familar with the SQL Server
landscape).

--
Chris Mullins

"LVP" <lvp_agentman (AT) hotmail (DOT) com> wrote

Chris,

What type of long term problems?
Can you share please I store images in a database.

Thank You,

LVP


"Chris Mullins [MVP - C#]" <cmullins (AT) yahoo (DOT) com> wrote in message
news:%23pCiTsmTIHA.1208 (AT) TK2MSFTNGP03 (DOT) phx.gbl...
In solutions that I've designed that have this as a requirement, I've
typically used Network Attached Storage.

In most cases, all the servers in the web farm are on the same gigabit
network segment, and the NAS (which is usually multi-homed) sits on that
same segment.

In most cases, storing files in the database will lead to long term
problems. As amazing as current databases are, they're still not file
systems...

--
Chris Mullins

"Redhairs" <redhair@u.s.a> wrote in message
news:%23GHoU4jTIHA.5788 (AT) TK2MSFTNGP06 (DOT) phx.gbl...
In a web farm environment, how to store the user uploading files for
future access?
Store them in db, local file system or centralized file server?

If trying to storing the file in local file system or centralized file
server, how the web server
receives the uploading file and sync to each web server's local file
system? or the centralized
file server?

Should I create only one folder or lots of subfolders to organize the
file structure for better access
from file system?