As with any museum, I think there is value in preserving things which exist only in singular physical form. As you point out, these things can only be seen by those with the means of making themselves physically present in the same location as the museum, which will severely limit the number of people with access, but there's not much to be done about that. If you want to see the Mona Lisa, you've got no choice but to go to the Louvre, the same as anyone else. But it's still more accessible than if it was in a private collection.
So in regards to a video game museum, even though the number of people who could access it is quite limited, there would still be value in it existing to preserve certain physical objects like prototype hardware or what have you.
But I personally see no point in putting things like books and magazines in museums. Their value is the information/content printed on their pages, not the pages themselves, so if a quality scan is publicly available, there would be little if any need to ever look at a physical copy. What a sad state of affairs it would be if the only way anyone could ever read Shakespeare would be for them to travel to whatever museum housed the first edition copy of whatever play and request a research grant to look at the pages behind closed doors. Sure, that's of value to some people, but to the world at large, it's the words themselves that have value, not the paper, ink and glue of the book they were printed in. So when speaking specifically of game magazines, sites like Retromags are FAR FAR more important than a museum could ever be for the purposes of preservation.