Skip to content

bitfield: unify order#12759

Merged
benpicco merged 3 commits into
RIOT-OS:masterfrom
miri64:bitfield/fix/bitorder
Nov 22, 2019
Merged

bitfield: unify order#12759
benpicco merged 3 commits into
RIOT-OS:masterfrom
miri64:bitfield/fix/bitorder

Conversation

@miri64

@miri64 miri64 commented Nov 20, 2019

Copy link
Copy Markdown
Member

Contribution description

Currently the bitfield type mixes up the order of bits: While the byte order is big-endian (most significant byte first), the bit order of each single byte within the bitfield is little-endian (most significant bit last). While this isn't a problem for most applications of the bitfield type it becomes one when sending the bitfield over the network (as done e.g. in the ACKs of Selective Fragment Recovery).

This change Currently the bitfield type mixes up the order of bits: While the byte order is big-endian (most significant byte first), the bit order of each single byte within the bitfield is little-endian (most significant bit last). While this isn't a problem for most applications of the bitfield type it becomes one when sending the bitfield over the network (as done e.g. in the ACKs of Selective Fragment Recovery).

This change unifies byte order and bit order to both be most significant bX first.

unifies byte order and bit order to both be most significant bX first.

Testing procedure

Users of bitfield (bloom filters, source address selection for IPv6, lpc2387 GPIO, ...) should still work as expected. I used bloom's unittests to test:

$ RIOT_CI_BUILD=1 make -C tests/unittests/ tests-bloom test --no-print-directory 
Building application "tests_unittests" for "native" with MCU "native".

   text	   data	    bss	    dec	    hex	filename
  46372	   4716	  47772	  98860	  1822c	/home/mlenders/Repositories/RIOT-OS/RIOT/tests/unittests/bin/native/tests_unittests.elf
/home/mlenders/Repositories/RIOT-OS/RIOT/tests/unittests/bin/native/tests_unittests.elf  
RIOT native interrupts/signals initialized.
LED_RED_OFF
LED_GREEN_ON
RIOT native board initialized.
RIOT native hardware initialization complete.

main(): This is RIOT! (Version: buildtest)
..
OK (2 tests)

Issues/PRs references

Needed for standard compliance in #12648.

Currently the bitfield type mixes up the order of bits: While the byte
order is big-endian (most significant byte first), the bit order of each
single byte within the bitfield is little-endian (most significant bit
last). While this isn't a problem for most applications of the bitfield
type it becomes one when sending the bitfield over the network (as done
e.g. in the [ACKs of Selective Fragment Recovery][SFR-ACKs]).

This change unifies byte order and bit order to both be most
significant bX first.

[SFR-ACKs]: https://tools.ietf.org/html/draft-ietf-6lo-fragment-recovery-07
@miri64 miri64 added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Area: sys Area: System labels Nov 20, 2019
@miri64 miri64 requested a review from cgundogan November 20, 2019 18:01
@cgundogan

Copy link
Copy Markdown
Member

Just for clarification: the issue you are facing only appears when you do a plain memcpy() of the bitfield, instead of copying each bitfield byte (which would circumvent the byte order issue) with a loop into the packet header, or am I wrong?

@miri64

miri64 commented Nov 21, 2019

Copy link
Copy Markdown
Member Author

Just for clarification: the issue you are facing only appears when you do a plain memcpy() of the bitfield, instead of copying each bitfield byte (which would circumvent the byte order issue) with a loop into the packet header, or am I wrong?

memcpy has nothing to do with this (the type is used verbatim in the RFRAG ACK). The issue is, that you start counting the bytes at "the back of the byte", while you start counting the bytes at the front of the bitfield which is not how SFR expects the bits to be counted. Sure, I could provide a separate bitfield module that counts them as expected or copy each bit by hand so the header is correct. The first wouldn't be very re-usable and a lot of code duplication, the latter not very efficient.

@cgundogan

Copy link
Copy Markdown
Member

Since you are now imposing a strict format of how the bitfield is internally represented, I very much would like to have a specific unit test for that .. to prevent future changes in this module to generate regressions.

@miri64

miri64 commented Nov 21, 2019

Copy link
Copy Markdown
Member Author

Will do.

Comment thread sys/include/bitfield.h
@miri64

miri64 commented Nov 21, 2019

Copy link
Copy Markdown
Member Author

I didn't notice there were already unittests for bitfield that are now failing 😅

@miri64

miri64 commented Nov 21, 2019

Copy link
Copy Markdown
Member Author

Will do.

Added unittests for missing functions and also fixed the existing unittests.

@cgundogan cgundogan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with this change. The new behavior is properly documented and unit tests were extended. ACK! No idea why travis fails with a docker error, though ..

@miri64

miri64 commented Nov 21, 2019

Copy link
Copy Markdown
Member Author

It's a cppcheck error ;-). The "docker error" is just the docker run that failed because cppcheck failed. Will address the cppcheck error

@miri64 miri64 force-pushed the bitfield/fix/bitorder branch from 002678b to 27510b7 Compare November 21, 2019 11:40
@miri64

miri64 commented Nov 21, 2019

Copy link
Copy Markdown
Member Author

Addressed and squashed directly.

@benpicco benpicco merged commit d86ca37 into RIOT-OS:master Nov 22, 2019
@miri64 miri64 deleted the bitfield/fix/bitorder branch November 22, 2019 21:38
@fjmolinas fjmolinas added this to the Release 2020.01 milestone Dec 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: sys Area: System CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants