How can I circumvent idiot web page designers?

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 11492
Joined: 24 Jan 2010, 23:23
Location: paused.undefined.exposed

How can I circumvent idiot web page designers?

Post by ChrisGreaves »

A link to the University of Toronto's faculty of Music web site:-
https://music.utoronto.ca/concerts-events.php?cDate=2018-03-29
The link is primed for a week from now.

This bugs the heck out of me:-
I can see that there are three events today (22nd) and only one on other days, somedays two events etc.
Walter Hall is just 15 minutes walk from my home, and these students of performance are all pretty good.
I would like to plan my weeks ahead by finding out what concerts are on at Walter Hall, as distinct from other locations, the dates and times.

The web page merely tells me, fo a single day, that there are "three concerts today", but it does NOT tell me the place, and it does NOT tell me the time.
To find the place and time I am forced to click on the link, make a note or save each individual page, then click to go back.

Why do web designers think that I have nothing better to do than click like a pigeon pecking at a lever?

I'd love to have an application that can d/l the web page and every subsidiary data that can be obtained by clicking on each individual's link.

At least then I could dash of a quick VBA to parse the data and find the stuff I need.

Failing that, I'd appreciate a clue as to how to interrogate the top web page (link given above) so that I can automate the process of downloading data.

For an example (IMNSHO) of a classic web page, see WholeNote. This page lets me see at a glance what days/times/places I should be.
(That I can parse it and drop items directly into my calendar is what I like about being able to obtain all the data).

:ranton: To my mind, people get confused about "data" and "information". A web page should deliver data to me; it should NOT deliver information that has been processed by the idiot web designer. :rantoff:

I am saying idiot, because the guy actually barged in on my enquiry with the librarian, and petulantly asked me if I had any idea how difficult it was to design a web page. I bit my tongue, thought of you guys, and decided not to tell him that I had designed and written my own web compiler after I'd seen how bad SaveAsHtml was in MSWord. After he left in a snit the librarian apologized to me. I still love librarians.

(signed) "lover of student musicians", of Toronto.
You can walk out of your house, but you always return Home

User avatar
John Gray
PlatinumLounger
Posts: 4764
Joined: 24 Jan 2010, 08:33
Location: A cathedral city in England

Re: How can I circumvent idiot web page designers?

Post by John Gray »

I also feel your pain.

I cannot count the number of times the monthly newsletter of a well-known cathedral indicates that some person or choir is giving a concert at a certain date and time but doesn't say what the concert is about, let alone the list of the pieces to be performed! :sad:
Mentioning this glaring omission to the newsletter editor brought the comment they were often not given the information most normal people would have wanted, so couldn't add it into the newsletter.

I would complain to the appropriate manager of the incompetent web designer, and continue until Something Was Done About It...
:bash:
John Gray

Snaccident - “the unintentional eating of an entire packet of biscuits".

User avatar
Jay Freedman
Microsoft MVP
Posts: 1102
Joined: 24 May 2013, 15:33
Location: Warminster, PA

Re: How can I circumvent idiot web page designers?

Post by Jay Freedman »

Follow these links (see what I did there? :grin: )

https://www.techrepublic.com/article/do ... wnthemall/

https://superuser.com/questions/274414/ ... d-from-one

There are many similar answers; I found these with this search.

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 11492
Joined: 24 Jan 2010, 23:23
Location: paused.undefined.exposed

Re: How can I circumvent idiot web page designers?

Post by ChrisGreaves »

John Gray wrote:I would complain to the appropriate manager of the incompetent web designer, and continue until Something Was Done About It... :bash:
Been there, done that. The dean of the school sent me an email saying soemthing along the lines of "look into it", but it sounded a lot like "Run along now, Sonny; we have important things to do here".

(signed) "glum" of Toronto
You can walk out of your house, but you always return Home

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 11492
Joined: 24 Jan 2010, 23:23
Location: paused.undefined.exposed

Re: How can I circumvent idiot web page designers?

Post by ChrisGreaves »

Thanks Jay.

This first one failed with a "408 error" on the UofT FoM web site. I will try it on another site.
And this one seems to require a utiliuty "wget".
I pasted the first command line as-is and my DOS window told me
"'wget' is not recognized as an internal or external command, operable program or batch file."

The conversation I had with what turned out to be the web page guy suggested that he had slaved long and hard to make this web site incredibly clever.
I suspect he is too clever to write a simple web page with links.

I'll keep trying.
Thanks again.
Chris
You can walk out of your house, but you always return Home

User avatar
Jay Freedman
Microsoft MVP
Posts: 1102
Joined: 24 May 2013, 15:33
Location: Warminster, PA

Re: How can I circumvent idiot web page designers?

Post by Jay Freedman »

https://www.gnu.org/software/wget/
GNU Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.
...
Runs on most UNIX-like operating systems as well as Microsoft Windows
Download the Windows version from https://eternallybored.org/misc/wget/

By the way, be sure to read the response by Pablo Halpern near the end of the thread on superuser.com. It points out a number of useful switches in the wget command line.

User avatar
John Gray
PlatinumLounger
Posts: 4764
Joined: 24 Jan 2010, 08:33
Location: A cathedral city in England

Re: How can I circumvent idiot web page designers?

Post by John Gray »

Yes, wget is a well-known utility - I use it (v1.17.1) in a BATch file every time I log on to obtain the (current) external IP address of my router from one of the WhatIsMyIP-type websites.
John Gray

Snaccident - “the unintentional eating of an entire packet of biscuits".

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 11492
Joined: 24 Jan 2010, 23:23
Location: paused.undefined.exposed

Re: How can I circumvent idiot web page designers?

Post by ChrisGreaves »

Jay Freedman wrote:Download the Windows version from https://eternallybored.org/misc/wget/
Hi Jay and John.
I must be doing something wrong. I'm probably tired or poorly fed.
WGget.png
I have downloaded four versions (including the 1.17.1 mentioned by John) and all four deliver the same result, an error in Copying.

I am d/l the 64-bit version because I am running the Win7HP 64-operating system.
The zip files are d/l into my B: drive, which is a SUBST drive, but I am extracting them from the path T:\Blotter\20180403\, which FWIW is a TrueCrypt drive.
I get the same errors when I make a folder C:\20180403\ and copy the zip files there before "Extract All".

If you can spot a simple "doh" mistake I'd be grateful; otherwise I think I'll try again tomorrow after I've eaten a couple of meals and slept overnight!

Cheers
Chris
You do not have the required permissions to view the files attached to this post.
You can walk out of your house, but you always return Home

User avatar
Jay Freedman
Microsoft MVP
Posts: 1102
Joined: 24 May 2013, 15:33
Location: Warminster, PA

Re: How can I circumvent idiot web page designers?

Post by Jay Freedman »

I don't see any "doh" but I didn't run into any trouble with either the download or the zip extraction. I've temporarily posted the contents of the zip file (for 1.17.1, 64-bit) on my own website for your convenience. Use these links:
https://jay-freedman.info/wget/wget.exe
https://jay-freedman.info/wget/wget.html (documentation)
https://jay-freedman.info/wget/COPYING.txt (GNU license)
https://jay-freedman.info/wget/wget.ini.sample.txt (sample ini file, commented)
https://jay-freedman.info/wget/wget.debug.exe (debug version of the exe, probably not useful to you)

(Note: The last three files are slightly renamed from what's in the zip file, to make them accessible without having to modify the MIME file on the website.)

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 11492
Joined: 24 Jan 2010, 23:23
Location: paused.undefined.exposed

Re: How can I circumvent idiot web page designers?

Post by ChrisGreaves »

Jay Freedman wrote:I don't see any "doh" but I didn't run into any trouble with either the download ...
Jay, thank you for your efforts!
I have now d/l the files from your site links.
(True Confession: I was so excited to d/l the exe that I immediately double-clicked on it and was terribly disappointed that it just went "FLASH" and disappeared. Then I slowed down my 71year old CPU and considered running it from a dos prompt ... :rofl: :laugh: )

I have now managed to d/l a page or several in PHP format, so I will go home tonight and read the html documentation.
The switches appear to outdo those for RoboCopy.
This could be a good thing or a bad thing (please see "71year old CP" above).

Again, many thanks. :cheers: :chocciebar:

Chris
You can walk out of your house, but you always return Home

User avatar
Jay Freedman
Microsoft MVP
Posts: 1102
Joined: 24 May 2013, 15:33
Location: Warminster, PA

Re: How can I circumvent idiot web page designers?

Post by Jay Freedman »

You're certainly welcome! I hope you're now able to compile the concert schedule you want.

I'm about to remove the files from my website, because I'm not an official distributor and because that isn't the most current version.

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 11492
Joined: 24 Jan 2010, 23:23
Location: paused.undefined.exposed

Re: How can I circumvent idiot web page designers?

Post by ChrisGreaves »

Well, this is promising. Here is a snapshot of an excerpt from the d/l Glen Gould School of Music.
Absolutely wonderful!
Now to tackle the dreaded faculty of Music of UofT.
Thanks again to all.
Chris
You do not have the required permissions to view the files attached to this post.
You can walk out of your house, but you always return Home