At that time our relations were going upwards, but since then they have been going downwards [downhill].
⇑ Translate the sentence above by Google ⇑
Related: downwards