__/ [ Satchel ] on Friday 17 March 2006 23:39 \__
> Roy Schestowitz wrote:
>> __/ [ Larry ] on Thursday 16 March 2006 13:18 \__
>>> I have just started using Debian but have been around computing for 20
>>> What I want to do is capture the market data my broker feeds to me via
>>> their trading page. the data is not streaming, but changes fairly often.
>>> The web page containing the data is constantly updating,
>>> except I want to be able to take a snapshot about 20 times per hour for
>>> the "put/call" prices at a specified time so I can have a historical
>>> timeline. The broker does not provide a historical/daily/minute updates
>>> in a file format I can use so it is up to me to capture this data and
>>> massage it into a format I can use.
>> Scrape it, then interpret and process it.
>> ,----[ Command ]
>> | man wget
>> If the data is provided in the form of feeds, you might also wish to parse
>> it using tools from the RSS 'family', e.g.:
>>> Several years ago I wrote some "c" file utilities that would parse out
>>> bad data from downloads but I don't think the above described application
>>> would be the same as this was kinda pre internet....still using Bulletin
>>> Boards back then with a 300 baud modem. Now, we have html,xml, etc and I
>>> was wondering how to get started capturing this information.
>>> I wish I could find a macro, as you see I don't know what is available
>>> but I have googled to death.
>> Let me know it you need further help because I have some useful scripts at
>>> Thanks in advance for any suggestions.
>>> Birmingham, Al.
>> Hope it helps,
> Thanks Roy,...I will be calling upon your advise
__/ [ Satchel ] added on Friday 24 March 2006 14:52 \__
> Ok...I was away for a while. You said "Scrape it". I suppose that is a
> script of somekind. I haven't a clue what you meant
"Scraping" is just a term, which often refer to the act of taking content
from one site and using it in another. It *might* also apply to the scanrio
where content is fetched in unexpected ways (e.g. spidering), as in this
case. Depending on the process/intent for which you fetch the data, I guess
the terminology (affected by and dependent on ethics) will vary. Scraping
has a very gross connotation.
Roy S. Schestowitz | while (sig==sig) sig=!sig;
http://Schestowitz.com | SuSE Linux ¦ PGP-Key: 0x74572E8E
3:40pm up 16 days 8:17, 8 users, load average: 0.98, 0.82, 0.81
http://iuron.com - next generation of search paradigms