I like this poem, the imagery of the swing is engaging and personal. But you ask for constructive criticism, so here it is.

First, I'd remove the parentheses and ellipses from the opening. I know you want to set it off, but you can more easily accomplish it with line spacing.

"When we grew" is confusing. While I suspect I know what you're saying, it does not help the poem to force people to read your mind.
Remove the semicolon after swing and get rid of the comma after heights.
Dreary is a somber word, dark and boring; perhaps you should try weary.
You could do without the ands, especially not to start a sentence with them.
Passable chorus, but the rhymes are forced. For instance, why would you "fight" for your love? There is no conflict introduced anywhere that would require you to fight.

^^Up the tree you went,
all on your own
To heights, we had never before known
My dear sweet love,
you lost your grip.
You could never return
from such a slip.

Here should be the heart of the emotional impact of the work. Here is where you should tie the title into the poem. What happened? This metaphorical climb up the tree and losing his grip: there's no emotional impact. Make it harsh or guttural, take out the serenity.

Remember I said at the beginning I like it. Don't take my criticism too much to heart, others will disagree. But the poem can be made much better.