Hello and congrats on your first coil! Yours looks better than my very first one, for sure. When I started monkeying around with pickups this forum didn't exist and there was very little information on pickup winding available as a whole. As a beginning winder, you are very very fortunate to have the internet and this forum as a resource for learning. Using the forums search function may help you cut down on the 'trial and error' part slightly, but don't expect any of this to come easy- things holding real value in life rarely do.
And speaking of easy, 42 is much easier to work with for a beginner and I think you're right to try a pair of 5000 turn coils - go for it!
A few tips:
1. Push your tensioning contraption aside for now, get your bare fingers on the wire, and begin to develop the tactile feel and control required for hand-tensioning. As a starting point: try taping a AA battery to the end of a 12" piece of coil wire, holding the wire with your fingers and gradually increasing your finger-pressure until the wire doesn't slip. Pay attention to what the pressure you're exerting on the wire feels like. Practice this, and then apply that same touch as closely as you can on the next coil you try. And along with that...
2. Slow your winder down. If your winder has a "sweet jesus, that's slow!" setting, use it. Concentrate on moving the wire back and forth in an even manner and hitting your desired number of turns as you do so, maintaining an even tension on the wire from start to finish. Resist the temptation to speed up until you're making nice looking coils at the slow speed consistently and have your finger tension correct for that speed. You need to develop a base-line for tension and this is, in my opinion, the smart way to do it. Once you're comfy and have a handle on things, then start ramping up the speed.
3. Despite the loose coil you created, I'd suggest winding the other coil and completing your humbucker build. If it works at the end, consider it a success! This will not only empower you, but it will also allow you to get the hang of soldering and the assembly process. I also hope you hook it up, plug into an amp, play, and listen closely to what you hear. Take notes. Love it or hate it, it will help you develop your ear regarding coil tension, creation, and form as it applies to what you want to hear from a pickup. If it sucks, don't be discouraged - just gut it and try again. And again. And again. Hard work and perserverance DO pay off, so stick with it and you'll start getting the results you want.