samedi 25 juin 2016
Download complete webpage after keypress in Java
I have a project that takes in an HTML file and parses it, and this works fine. I'm now trying to automate the process so that it happens on a schedule. I have Jenkins set up on Ubuntu Server, and the process works well, except for the actual creation of the HTML file - that has to be done manually. It's unfortunately not as easy as a CURL command because the webpage has some JavaScript for that "never ending content" feature - in order to actually get every link from the site, you have to continually scroll down until it loads them all.
So, I have a relatively simple process that I'm now trying to automate, hopefully with Java so that it can simply be the first method I call. Because it's run on Ubuntu Server, it definitely needs to be able to run "headlessly" - so a macro that opens the browser won't work.
The process:
Open browser
Navigate to URL
Hold down space bar for a while to continually scroll down
After I notice that no more content is being added (this can just be set to 'after 2 minutes'), hit CTRL+S and save the page to a network location
Hit build on Jenkins, which executes the Java program, which reads from the location and parses the file
Because the saving is manual, building in Jenkins has to be manual and can't be put on a schedule. How can I automate this process with Java?
Inscription à :
Publier les commentaires (Atom)
Aucun commentaire:
Enregistrer un commentaire