TIL, 2017-07-03

• It’s hard to do the limiting N+1. Just go for caching.

Chat with Mark on Scraping

• Watir: Web Application Testing in Ruby. Slow Internet – results in visible and Net::ReadTimeout errors. Headless is faster, but you have to assign the user-agent. No prod for Watir since it leaks memory.
• Mechanize does not run JS. But it is faster than Watir.
• Check out the Resources log in Inspect Element to see the endpoints that make sense.