Seems to serve a couple of functions. The whistle identifies her with the bullfinches in that Tess too is trapped and dependent. Also, it has seductive overtones.
1. From:
http://www.pinkmonkey.com/booknotes/barrons/te ssurb3.asp
Mrs. d'Urberville demands that Tess (often described as a trapped bird) learn to whistle so she can train the Lady's bullfinches to sing. Tess practices to no avail until Alec teaches her how to purse her lips. Frightened that she'll lose her job, she lets him instruct her in an art that resembles kissing. He seems kinder to Tess now, and she begins to trust him a little. However, it is less a matter of free-willed trust than the fact that Tess, like Mrs. d'Urberville's birds, is dependent upon her keepers.
2. From:
http://www.linguist.org.cn/doc/uc200709/uc2007 0905.pdf
Other instances of symbolism prevail in the book. One is the strawberry. Alec forces Tess to eat on her first
visit to Alec�s home, where Tess is showed around the house until they come to the green-house where the
strawberries are grown. The fact that the strawberries are made ripen before the season symbolizes that Tess, like
the strawberries, is also to be forced to �ripen� by the external forces represented by Alec�s seduction of her later.
The scene of Alec�s teaching Tess how to whistle serves the same function. The line of tune is obviously seductive,
but the allusion is unknown to Tess. Just like her learning to whistle without knowing what she is whistling, she is
seduced by Alec without guarding on his vicious attempts on her.