[Melbourne-pm] Designing modules to handle large data files

Kostas Avlonitis dsk_gr at hotmail.com
Tue Aug 24 22:39:11 PDT 2010

>> I am lookin' right at you, here.)

Wow. Probably some background I'm not aware of.
However I don't know when was the last time anyone convinced anyone else 
using irony and personal calling-out as a method - unless they're 
playing for the audience - or perhaps it didn't come across as intended.

I think Sam's concept of a flat file is valid for a single-user setup 
even with even a medium volume of data.
However there are potential problems:
Normalisation and maintenance may become issues if the types are not 
strictly handled by the app or in a loose multi-programmer environment. 
Also, as previous posters said, probably not easily scalable as the 
back-end of even a medium multi-end-user setup (locking records, 
cross-referencing it with other data, adding fields, indexing, massive 
data-growth, multiple data-maintainers, reporting additions etc). DBs 
are a pain but not as much a pain as maintaining files in my experience.

I guess it depends on the scale and breadth of the application, but I'm 
putting my vote on the side of when-in-doubt use a DB.


More information about the Melbourne-pm mailing list