During E16's beta testing period, I went through in RPG Maker and noted down all of the relationship points.
A few things of note:
1. The game will not tell you when something you do effects relationships. This info can only be found by looking through events in RPG Maker.
2. When gaining a point with someone for speaking while teamed up, or losing a point by leaving in an area where you're teamed up with them, this only happens once. Meaning you can't speak to them repeatedly for infinite points, or if you have to leave more than once you'll only lose (a) point(