I had to use a custom CSS grid for the basic task to create this UI because semantic ui assumes that I need padding and margins in the grid. I'm not sure if it's possible, but I think the grid system should either include a nopadding class/option or not assume that it should add padding. If no option to disable this is present I think the padding should be created by containers/segments, but not the grid itself.
TLDR: Grid shouldn't assume padding, but segments and containers should. Otherwise a class would be nice.