AppleScripting Safari Downloads
Posted February 7th, 2007 @ 12:16pm by Erik J. Barzeski
I've done some testing AppleScripting Safari image downloads from a "members" site (a photography club). Images are always of the form http://server.com/[prefix]xx[suffix].png, with "xx" being a number like 01 or 43. Currently I have a script capable of loading all of the images:
- I log in to the site and navigate to the images I want to download.
- I run an AppleScript which prompts me for the number of items, the prefix, and the suffix.
- The AppleScript builds
xURLs and then usesopen locationto open them all. - I manually drag and drop the images to a folder I've created.
The last step can take some time. Unfortunately, Safari's AppleScript dictionary doesn't allow for downloading images. The standard save document 1 approach doesn't work either - Safari attempts to save a page archive instead of the actual PNG file.
I did some Googling and discovered that "URL Access Scripting" has a download method, but this method can't get past the site's membership login requirements (and this is not a login which can be accessed via "http://username:password@server.com").
So, what to do? I'm opposed to using another browser for this functionality, and I don't believe tapping into Safari's JavaScript capabilities (via AS) will work. Can images from a members/login site be downloaded from Safari via AppleScript?
Donate Life
Posted 08 Feb 2007 at 2:08pm #
You might try using the unix tool wget ? It supports credentials via HTTP and/or cookies from a text file. man wget for more information.
Posted 08 Feb 2007 at 2:19pm #
bgk said on February 8, 2007:
Surprisingly, wget doesn't appear to be a standard component of Mac OS X, though curl is. I haven't looked into using curl for this, but if it involved much monkey business each time, it wouldn't be any better than manually dragging the images to a folder.
Posted 08 Feb 2007 at 2:47pm #
Erik J. Barzeski said on February 8, 2007:
curl is really good at dealing with patterns like this.
curl -O "http://www.site.com/file[01-25].jpg"
will download the 25 files to the "file01.jpg" through "file25.jpg" to the current working directory.
Posted 08 Feb 2007 at 3:27pm #
Clark Cox said on February 8, 2007:
I know it is. But curl will simply download a "please login" page. Please note the "membership/login" requirements mentioned above.
Posted 08 Feb 2007 at 4:30pm #
If the site sets a cookie once you login, you could likely login once from Safari, pull the relevant cookie out of ~/Library/Cookies/Cookies.plist and pass that to curl via the "--cookie" parameter.
Baring that, you could probably have curl send the login form data directly using the "--data" flag.
Posted 08 Feb 2007 at 5:05pm #
Clark Cox said on February 8, 2007:
Looking through a 2.5 MB file every time I want to do this is no easier than manually downloading.
Clark Cox said on February 8, 2007:
I don't think that would work either. I don't think curl can maintain the "logged-in-ness" of the session.
Simply put: Safari should let me do this.
Posted 08 Feb 2007 at 6:25pm #
Erik J. Barzeski said on February 8, 2007:
The "--cookie-jar" option looks promising. I'd imagine that if the login info is stored in a cookie, using this option will allow the "logged-in-ness" to persist across invocations of curl.
Erik J. Barzeski said on February 8, 2007:
Oh, I agree wholeheartedly, just trying to help find a workaround.
Posted 08 Feb 2007 at 7:31pm #
I'm not sure if either of these are helpful but...
http://bbs.applescript.net/viewtopic.php?id=15805
http://automator.us/examples-02.html
Posted 08 Feb 2007 at 7:43pm #
Andy Fragen said on February 8, 2007:
That's where I found out about "URL Access Scripting," but as I said above, it can't handle the login info.
Andy Fragen said on February 8, 2007:
Automator seems less capable than AppleScript, though I'm sure the two could somehow be paired… perhaps the AppleScript could create a new window, load the next image, call an Automator action to download the image, and move on? Is that possible?
Or, better yet, a list of URLs could be created for the "Download URLs" list. But how can the list be created in AppleScript (as is currently done)?
Hmmm… this looks promising, but I'm not getting my hopes up. I hadn't thought of Automator. Thanks.
Update: This, too, simply downloads the login page.
Posted 09 Feb 2007 at 7:57am #
Did you try the --cookie-jar option? I would think you could write a shell script, which did:
1. curl: call the login script with your credentials, and ask to save the cookies created in a named cookie-jar file
2. curl: ask to download a sequence of files with the --cookie option providing your login state
The shell script could handle options for where to store the downloaded files, and your sequence/prefix/suffix params. At which point you could wrap this in an AppleScript if need be to be able to call it through Safari's script menu (or hook up a trigger in QuickSilver).
Posted 09 Feb 2007 at 12:44pm #
Using System Events scripting, you could paste the URLs one at a time into the safari address bar and then have it type option-return to download rather than view.
Posted 27 Apr 2007 at 5:59pm #
with curl you can use -c to write a cookie file received, -b to read a cookie file, -d to fill out a form like this:
curl -c '/Users/firstnamelastname/Documents/CurlCookies/nameofsite' -d 'login=joe&pass=1234' http://www.nameofsite.com/login.php
now you are logged in, and you have a cookie stored locally (I made a folder in my documents folder). you can then read that cookie.
curl -b '/Users/firstnamelastname/Documents/CurlCookies/nameofsite' http://www.nameofsite.com/whatever.jpg