Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

157
Views
Fast Way to Get Data from Website in Java

I'm trying to write a program that utilizes the Wikipedia API. As far as I can tell, the simplest way to use the API is to access an HTTP page with the requested command, for instance, this finds all links on the "Apple" wikipedia article. I want to implement commands like these into my Java program so I created the following snippet to fetch the data from a HTTP page:

    URLConnection connection = null;    // Connection to the URL data
    InputStreamReader iSR = null;       // Stream of the URL data
    BufferedReader bR = null;           // Reader of URL data
    URL url = null;                     // URL based on the specified link

    // Open the connection to the URL web page
    url = new URL(link);
    connection = url.openConnection();

    // Initialize the Readers
    iSR = new InputStreamReader(connection.getInputStream());
    bR = new BufferedReader(iSR);

    // Fetch all of the lines from the buffered reader and join them all
    // together into a single string.
    return bR.lines().collect(Collectors.joining("\n"));

This works fine for fetching data, however, it is very slow. Each fetch takes around half a second, which is unacceptable for my program, especially since processing the whole downloaded file only takes around 1/1000th of a second. Is there any way that I can somehow quickly download these small files?

over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

The fastest way, if you don't mind not having the most up-to-date information (of course you could devise a way to keep the data up to date), would be to get a dump of the data.

This would allow you to create your own server that could return pre-formatted data, as well as returning multiple data items with a single request, making it a lot faster than parsing HTML from multiple requests.

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!