I built 7 expresses this year and my first was similarly slow.
By my last 3 or 4 I was cranking one out per night, pretty easily ... or two fairly short nights ... maybe 2-3 hours total.
I would put a completed (and tested) express board out on the bench, where I could view it as a reference. Mainly this was helpful to see the color patterns of a few things as needed. After a while I just started to realize where everything went.
In general, the first half of the build was the small parts, incl. sockets ... and getting all those small parts installed and soldered on. Some nights I would take a break or stop at that point. Then in a second phase or night ... I would do everything else ... and those larger parts would go faster by comparison.
To me, the real speed came with my own comfort of knowing what parts went where ... knowing how to quickly identify them ... and being able to look at the reference real quick when in doubt. Obviously following the manual for the most part, but not having to reference it ... made things go quicker. I would never advise anyone to skip the manual if they are not feeling comfortable with building them yet. For me, I referenced the manual a little on #2 ... and on #3 thru #7 ... hardly at all. They all worked.