Home Closet Sensors for content monitoring on a Web server

Sensors for content monitoring on a Web server

by admin

Thinking about "making it cheaper" to monitor the updates of desired web pages, I formulated the concept of "sensors". I haven’t seen any mention of it before, so I’m publishing it here and now.
The essence of the concept lies in a special modification of the software Web-server, ie create and connect a software module, which will "watch" for incoming HTTP-requests and "listen" to the answers the server. Each request (based on information http-headers can be made up a unique identifier of the requested resource) must be matched against the response (can be processed by the algorithm md5 or similar) server. In the next step, this pair is searched for in the already accumulated Web-server database of resource meta-descriptions. If changes are detected, it is suggested to send some meta-description of found changes to the main public server immediately (or as the Web-server is unloaded) (for example, using GET request, in which meta-information will be encoded as parameters). Actually for this reason (the analogy with various sensors used in technical practice is obvious) I decided to call the module "sensor". And then, from the public server, the information about updates to the page or any other content can be obtained by any "consumer" on demand or in some other way. I think the idea is clear.
This concept was outlined a relatively long time ago (in 2005) in his dissertation True, the emphasis of the dissertation was on creating a system for modeling the process of monitoring websites and so the concept of sensors was not discussed thoroughly. Now, while I am sitting without work, I decided to involve the hubro community in my research and hope to get an interesting (at the time I couldn’t get through to interested people) response. Thanks in advance for your participation in the discussion!
Zy : My simulation experiment showed greater efficiency (read optimality in terms of "freshness" and "traffic savings") of the sensor-based monitoring system than the classical monitoring system (e.g., search engine robots).

You may also like