While it may be intuitively useful to simply frame the issue as derivative of simple single bond motifs, it is actually only the beginning of the discussion. The scientific model you described here is just too simplistic and at times the language you used here is also somewhat incorrect. I will explain.
There are no molecules in diamond or graphite. Those are networked or sometimes crystalline or semicrystalline materials, and under certain conditions and with certain structures, some allotropes of carbon (like when you have graphite, nanotubes, or buckyballs) can demonstrate delocalized electron bonds across a lattice of atoms. The molecular orbital approach that is useful in singular molecules thus becomes more extended and sophisticated by material scientists into what is often called a “density of states”. Then, after you know something about the density of states you add light to that density of states, and try to propagate it with equations derived from Maxwell’s Equations and then things become even more complicated, and thus fall outside of guidance of molecular orbital theory.
The total (often impure) band structure (reflecting the density of states) of a regularly arranged (on an atomic level) material and how light interacts with that band structure is what gives a solid material its sophisticated color. This is inherently difficult to model with consideration to irregularly placed impurity atoms; and, the model that material scientists use to predict color and appearance does not always involve absorbing photons…sometimes photons are scattered, sometimes they are re-radiated, and sometimes they are thermally generated by black body radiation (which involves mere kinetic vibrations of atomic material and can be independent of the light you hit the material with). It’s therefore not really the mere existence of individual Pi bonds an Sigma bonds summed together, it’s the way in which electrons populate the available electron density of states at a temperature and pressure relative to the way in which light can permeate or navigate those density of states. This also incorporates the notion of a “skin” depth of a material’s crystal where the density of states has to contend with altered surface electron states which can cause reflections and other color properties…we have to worry about surface states more so with other materials, for another example, nanocrystalline quantum dots.
In summary, light scattering, light adsorption, etc, all of these physical phenomena are very complicated processes (often governed by an interplay between quantum mechanics and maxwell’s equations) regulated by how the density of states (if applicable, i.e. for extended materials) across the material disturbs the propagating electromagnetic field found in light.
You might look up the “band gap” differences (gaps of energy within a density of states) between classes of insulators, a conductors, and a semi-conductors to see that often there are virtual states and impurity states within the density of states that mess things up for simple light propagation models…even in things like allotropes of carbon.
Thanks for stimulating the topic though. Science is deep. 
