Accelerating the Internet in the presence of Big Data: Reducing user delays by leveraging historical user request patterns for web caching

• Published in: Revista de gestão da tecnologia e sistemas de informação 2019, Vol.16, p.1-10
• Main Authors: Kumar, Chetan, Marston, Sean
• Format: Article
• Language: eng ; spa
• Subjects: Artificial intelligence; Autonomous vehicles; Big Data; Caching; Communications traffic; COMPUTER SCIENCE, INFORMATION SYSTEMS; Consumers; Data collection; Data management; Decision making; Electronic devices; Internet; Internet access; Internet of Things; Real time; Servers; Social networks; Velocity
• ISSN: 1807-1775; 1809-2640; 1807-1775
• DOI: 10.4301/S1807-1775201916006

Approximately 4 billion people have access to the Internet, additionally 23 billion devices are connected as of 2018. This has allowed for a substantial growth in data collection which has allowed for Big Data to flourish. The continued increase in user, devices, and Big Data usage has created a significant intensification in Internet traffic. This in turn has the potential to increase user delays when accessing data on the Internet. There are a number of ways to help reduce user latency, web caching is able to reduce web user delays in addition to reducing network traffic and the load on web servers. In this study we propose a proxy level web caching mechanism leveraging historical web patterns to help reduce user latency and accelerate the Internet. In addition we survey the state of the art of other caching approaches. Our investigation shows there is useful potential for effective proxy caching mechanisms that exploit historical request patterns to significantly reduce delays for web users if they were to be deployed in large scale networks in this Big Data era.