I haven't done it but more b/c the side of refrigerators is pretty ugly to look at. Can't imagine why anyone would forgo a panel to look at the side of a fridge.... but to each their own I guess

As far as support goes, if your well fastened in the back and the wall side, and the cabinet is built well it should be fine. And if your building them as it sounds like you are, you could always make the side panel integral to the frame..... that's how I do side panels on more expensive projects

good luck,
JeffD