Since the cube shares one of the dimensions of the rectangular solid, it must have a side of 4, 5, or 8. However, if its side is 5 or 8, it won't fit entirely within the solid. Since one of the lengths of the solid is 4, all of the lengths of the cube must be 4 or shorter. Thus the side of the cube is 4, and the volume of the cube is 4^3 = 64.
The volume of the solid is the product of the dimensions: (4)(5)(8) = 160. We're looking for the ratio of the volume of the cube (64) to the volume of the solid that is not occupied by the cube-- that is, 160 - 64 = 96. The ratio of 64 to 96 can be simplied by dividing both terms by 32. The result is 2 to 3, choice (A).