Magpie Pricer

Created: Jan. 28, 2018

Technologies used

I had a lot of books to sell. Hundreds, and I wanted the best price for them from 3 websites that bought 2nd hand books. The thing was that the price offered for each book could vary wildly. Was there a faster way than manually submitting each ISBN to each site to see what the best price was?

Time to get Python on it. Using Selenium I automated a web browser to submit a list of ISBNS, figure out the best price and build a basket for each site with the end result being the best price possible.

It worked, and it didn't take too long to do it - but it was finicky.

Turns out these sites didn't really like automated web browsing and I found that my scripts would break any time a site was updated as they looked for different HTML/CSS markers. The code was brittle and unreliable. I'd built something, but it probably wasn't worth the effort.

It was my first time working with HTML though so it was a learning experience, and Selenium is a great package to know for full end to end web testing of applications.

What I learned through this project

  • Automating a web browser with Selenium
  • That getting data by parsing html is not so easy
  • That just because you can doesn't mean you should