Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Need advice on a project I am trying....please help

  • Subject: Re: Need advice on a project I am trying....please help
  • From: Roy Schestowitz <newsgroups@xxxxxxxxxxxxxxx>
  • Date: Fri, 24 Mar 2006 15:42:30 +0000
  • Newsgroups: alt.hacker
  • Organization: schestowitz.com / MCC / Manchester University
  • References: <5EdSf.6$Qh6.4@fe02.lga> <dvf01a$ir8$4@godfrey.mcc.ac.uk> <DQHSf.278$qp3.198@fe05.lga>
  • Reply-to: newsgroups@xxxxxxxxxxxxxxx
  • User-agent: KNode/0.7.2
__/ [ Satchel ] on Friday 17 March 2006 23:39 \__

> Roy Schestowitz wrote:
>> __/ [ Larry ] on Thursday 16 March 2006 13:18 \__
>>> I have just started using Debian but have been around computing for 20
>>> years.
>>> What I want to do is capture the market data my broker feeds to me via
>>> their trading page. the data is not streaming, but changes fairly often.
>>> The web page containing the data is constantly updating,
>>> except I want to be able to take a snapshot about 20 times per hour for
>>> the "put/call" prices at a specified time so I can have a historical
>>> timeline. The broker does not provide a historical/daily/minute updates
>>> in a file format I can use so it is up to me to capture this data and
>>> massage it into a format I can use.
>> Scrape it, then interpret and process it.
>> ,----[ Command ]
>> | man wget
>> `----
>> If the data is provided in the form of feeds, you might also wish to parse
>> it using tools from the RSS 'family', e.g.:
>>         http://magpierss.sourceforge.net/
>>> Several years ago I wrote some "c" file utilities that would parse out
>>> bad data from downloads but I don't think the above described application
>>> would be the same as this was kinda pre internet....still using Bulletin
>>> Boards back then with a 300 baud modem.  Now, we have html,xml, etc and I
>>> was wondering how to get started capturing this information.
>>> I wish I could find a macro, as you see I don't know what is available
>>> but I have googled to death.
>> Let me know it you need further help because I have some useful scripts at
>> hand.
>>> Thanks in advance for any suggestions.
>>> Larry
>>> Birmingham, Al.
>> Hope it helps,
>> Roy
> Thanks Roy,...I will be calling upon your advise

__/ [ Satchel ] added on Friday 24 March 2006 14:52 \__

> Ok...I was away for a while.  You said "Scrape it".  I suppose that is a
> script of somekind.  I haven't a clue what you meant

Hi Satchel,

"Scraping" is just a term, which often refer to the act of taking content
from one site and using it in another. It *might* also apply to the scanrio
where content is fetched in unexpected ways (e.g. spidering), as in this
case. Depending on the process/intent for which you fetch the data, I guess
the terminology (affected by and dependent on ethics) will vary. Scraping
has a very gross connotation.

Best wishes,


Roy S. Schestowitz      |    while (sig==sig) sig=!sig;
http://Schestowitz.com  |    SuSE Linux    ¦     PGP-Key: 0x74572E8E
  3:40pm  up 16 days  8:17,  8 users,  load average: 0.98, 0.82, 0.81
      http://iuron.com - next generation of search paradigms

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index