I think I have this only after getting patch ...14 from Steam. Sometimes my influnce drops to the negative numbers way down, like -121 million. Largest drop was to -1.x billion. Also the bar on the bottom-right graph displays it to the left. My char is a level 64 warrior (loner challange) if it is important.
Sometimes it jumps back to positive, later on.