In my experience, I continually kept at the theory part while learning. I would always recommend trying to understand what and why you are doing something, in any endeavor.
That being said, it is best taken in small bites. Justin's book is concise and gives the nuts and bolts. I had another theory book, centered around guitar, more in depth. It was months before I learned much past what scales are and how to construct major and minor chords. The more "advanced" parts usually came as sudden revelations. It just comes.
You don't have to even know this to start, for quite a while. Much of the groundwork in learning guitar is learning the shapes of the handful of chords that get you at least 75% of the way, almost 100% of the way to enjoying it and playing songs. It is developing good technique, posture, hand position, avoiding bad habits, and developing muscle memory and speed. It is a pretty full plate for a while.
So, I would keep the theory around, for times to read and experiment. Won't hurt. But the real work is the beginner's course.