An Alternate Downloading Methodology of Webpages

We propose an advanced method for downloading Webpages from the Internet. In this technique, the whole system is considered as a bundle of crawlers which have been created dynamically at execution time. Numbers of crawlers are used depending on the requirement of downloading Webpages. The software m...

Full description

Saved in:
Bibliographic Details
Main Authors: Kundu, A., Pal, A.R., Tanay Sarkar, Banerjee, M., Mandal, S., Dattagupta, R., Mukhopadhyay, D.
Format: Conference Proceeding
Language:English
Subjects:
Artificial intelligence
Bandwidth
Crawlers
Earth
Educational institutions
Hierarchical downloading
Information retrieval
Internet
Multi-downloading
Uniform resource locators
Web sites
World Wide Web
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose an advanced method for downloading Webpages from the Internet. In this technique, the whole system is considered as a bundle of crawlers which have been created dynamically at execution time. Numbers of crawlers are used depending on the requirement of downloading Webpages. The software module which interacts with WWW to search one or more Webpages is known as crawler. The numbers of crawlers are generated using the hierarchy structure of the Web server from which the data would be downloaded. Webpage downloader is an important issue for downloading Web documents from the Internet to facilitate a Web user in terms of knowledge gathering. This type of downloaders are very popular in the 'Information Technology' field. All kinds of public data, accessible throughout the world without any authentication, can be retrieved any time from any geographic location using the downloading methodology. Typically, a downloading technique has been utilized to accumulate Webpages of different domains within a single computer machine one at a time. So, our aim in this paper is to show an advanced technique for downloading a lot of related Webpages with a minimum effort and time using hierarchical downloader consisting of several dynamic crawlers.
DOI:10.1109/MICAI.2008.13