Solving Times, Day by Day

A gentleman by the name of David Kaleko has analysed the solving times cited on, to test the assumption that the puzzles get harder as the week progresses. David has published his findings and outlined his methodology in a fascinating blog post. I suspect that some readers of might find his conclusions to be of interest.

Well done, David, and thank you!

6 thoughts on “Solving Times, Day by Day”

  1. Indeed it was very interestingly done…..although leaving out the people here in the comments section ruined it for me 🙂 And to defend Bill – he actually did the puzzle the day he switched formats. I guess it just got lost in translation.

    I’d be very curious to see what the same study would do with the NY Times posts which to me seem to have more variation in difficulty. Also – I’d be curious how the errors are spread out – as few errors as Bill ever has, he does have a few more over on the NY Times grids.

    One other note as to the outliers Mr. Kaleko ends up throwing out – it always seemed to me that Bill’s times were slightly higher whenever he was on vacation – be it Ireland, road trips or whatever. I doubt anyone would want to try to look into that, but I wonder how many of those results wound up being outliers.

    Thursday as the degree of difficulty of a Sunday sounds about right too.

    Bill you also realize that the only way this study works is assuming you’re a machine like entity that has constant functionality…which although a little dehumanizing, is probably pretty close to accurate 🙂

    Thank you, David and please visit us sometime.

    1. You make some very good points, Jeff!

      I think it’s fair to say that the outliers you mention were indeed closely correlated with celebratory occasions 🙂

      I too would like to see an analysis of NYTimes puzzle solving times (I’ve been recording my times on for many years). I’m will to bet that Sunday’s NYTimes puzzle wouldn’t show up as the slowest to solve, with Saturday and maybe even Friday puzzles taking longer.

  2. While I would normally just post on the same day, this seems special enough to comment on. Anyhow, while confirming what I’ve already surmised (*), I did find a formal analysis pretty interesting and would love to see the same tests run on other puzzles like the NYT, WSJ (which by experience I can state varies pretty wildly, though only has a 11 month full sample), Newsday (which definitely has the hardest Saturday puzzle by all accounts), CrosSynergy etc.

    The only question/problem I would put to Mr. Kaleko about his approach would be the presence of errors in terms of factoring difficulty, as he aims to do. It wouldn’t be hard to scrape this data from Bill’s site if he can already scrape times. Only thing would be weighting them into the time, which wouldn’t be hard (maybe 15 seconds additional per or something like that). Anyhow, a quite interesting read.

    And thank you, Bill, for posting it.

    * – I used Bill’s times a lot to try to gauge difficulty of puzzles while I was trying to find my footing in this in knowing how difficult the puzzles are I’ve been doing, especially when I completely blow it on one like last Thurs syndicated NYT – the only question I would have out of curiosity would be his times on some of Mr. Shenk’s “best shots”.

  3. @Bill Butler
    I mentioned the Word Play book in the current puzzle thread for today. The most relevant quote in it to what I see around here is (guess what?) – “an analysis of NYTimes puzzle solving times”. On p159, it’s stated that these are measures based off the NYT test solvers, the NYT online solvers, and aggregate ACPT data. They break it down further based on estimated skill level of the solvers (“Beginner”, “Intermediate”, “Expert”). Given the original link, I’ll just quote their average “expert” metrics (all in minutes, progressing days from Monday to Sunday):


    Of course, this was 2006 data, which I’m sure has changed if they ran the same data they have in 2016, since I’m (pretty) sure the editing has changed in the 10 year period.

    FWIW, they do not provide “Beginner” times for Thu/Fri/Sat and “Intermediate” times for Sat because they do not expect those puzzles to be finished by those classes. I can post the other metric sets if people want to measure.

  4. Re puzzle Jan. 28, 2019: Why is AUTO the prefix for CORRECT?
    (…maybe I should have checked to see if there’s a word, “autocorrect”, which is a “new world word” (!!) not familiar to the dinosaurs of my era…..)

Leave a Reply

Your email address will not be published.