event driven programming...

[Chuck Wintle]

How is this done and in which language? The event is the arrival of a file in a particular folder and the response is to copy the file to another folder and delete the file immediately upon being copied. I think C++ would be the best for this?

[Graham Wintersgill]

Chuck

I take it this is for your Windows 7 machines? If you have got the batch file to work then in windows help search for Scheduler and it will detail how to set up a schedule which you can run as often as you like. My only concern is it may try to copy a file that is only half created. Is the job to produce the figures run from a batch that you could add your batch to? The other alternative is to create the file in another directory and then move (not copy) the file to the pick up directory. THere may be a way to get windows to react to a new file being created but I am nt sure how that works.

Regards

Graham

[Matt Meiser]

.Net has a DirectoryWatcher object that's perfect for this. I've used it a few times.

There's a gotcha for this though--does the application writing the file open the file and stream data to it over time? I ran into this with the software for a gauge one of my clients uses. Even if the streaming only takes a second, that's a LONG time in software! I just had DirectoryWatcher start a timer. After 2s (user-configurable but the test always takes <2s) the file is processed.

[Chuck Wintle]

.Net has a DirectoryWatcher object that's perfect for this. I've used it a few times.

There's a gotcha for this though--does the application writing the file open the file and stream data to it over time? I ran into this with the software for a gauge one of my clients uses. Even if the streaming only takes a second, that's a LONG time in software! I just had DirectoryWatcher start a timer. After 2s (user-configurable but the test always takes <2s) the file is processed.
I don't think so...although it might takes a few seconds to put the file in the folder from the program that creates them. I would need to build in a small delay to be sure the file is completely written before proceeding to the copy.

[Chuck Wintle]

Chuck

I take it this is for your Windows 7 machines? If you have got the batch file to work then in windows help search for Scheduler and it will detail how to set up a schedule which you can run as often as you like. My only concern is it may try to copy a file that is only half created. Is the job to produce the figures run from a batch that you could add your batch to? The other alternative is to create the file in another directory and then move (not copy) the file to the pick up directory. THere may be a way to get windows to react to a new file being created but I am nt sure how that works.

Regards

Graham

yes it is a windows 7 machine and yes I would need to wait until the file is completely written before copying.

[John Aspinall]

The choice of programming language should be almost "whatever you are most comfortable with". Most language runtimes, or libraries, should provide the functionality you need.

As Matt has already mentioned, though, doing this right requires some defensive programming. Arrival of a big file should not be seen as one single event. Underneath the covers, there may be several events.

creation of the initial (maybe empty) file
one or more additions to the file
final closing of the file by the original writer-client

IF (and this is a very important, and big "if") you can be notified of that last event, then bingo - that's the event you want to trigger on, and you should be all set. If this is a high-usage scenario, you might still need to guard against arrival of a second file while you're copying the first, but perhaps that's overkill for your situation; I don't know.

But I can tell you from experience that the operating system usually considers the first of those events (initial creation of the file) to be the most significant event, and if the file is big enough (or the writer is slow enough), that's not the event you really want to trigger on. If I were doing this "right" (i.e. for pay, with code review by my professional peers), and I had only the first event to trigger on, then I would:

have that first event trigger a wait loop with an "is the file complete?" test to continue
only when the file was complete, copy and delete it


"Parallel programming - it's just one thing after, before, and simultaneous with, another."

[Myk Rian]

only when the file was complete, copy and delete it
Isn't there an EOF embedded in the file to signal that?

[Matt Meiser]

Its not necessarily reliable.

[John Aspinall]

Isn't there an EOF embedded in the file to signal that?
Again, Matt beats me to it, but no, there isn't. In most filesystems, the EOF (End Of File) is not in the file. The filesystem knows exactly how long the file is, and when a reader asks for (e.g.) the 43rd byte of a 42 byte file, the filesystem returns an EOF to the reader. So a reader will always get an EOF when they read past whatever is currently written. The check for "is the file complete?" is usually done with some application-level knowledge, for example

these files are always 4096 bytes long
these files always end with "love and kisses, Mom"
...