When I tried to follow this on Ubuntu I have always had some incompatibility errors around Numpy. Eventually I found out that there is a pip bug that links pycocotools against the newest Numpy binary even if you specify that you want to use Numpy 19.05.xx - which is the version compatible with the TF object detection libs.

https://zby.github.io/ML_Adventures/

I published my installation script: https://gist.github.com/zby/e3341751f54bc92ebcc1d9da08a9459a