See attached file.
In this problem you are given almost everything you need and that makes the problem a trivial math exercise. So, I think it is better to start from scratch and only assume the given formula for the entropy (which can be justified by saying that to have a longer string means that the polymers in it have to be more aligned so you expect the entropy to decrease to first order as L increases).
We want to find the thermal expansion coefficient, which is defined as 1/L dL/dT at constant tension F (I'll write partial derivatives as ordinary derivatives and specify what other variable is held constant). We want to show that it should be negative. This can be derived from the fundamental thermodynamic relation. For a gas this is:
dE = T dS - P dV
We need to know the analogue of this equation for a string. The T dS term is the heat supplied to the gas and in case of a string it is, of course , the same. The P dV term represents work done by the gas, which comes at the expense of the gas so it enters the equation with a minus sign. In case of a string you have instead of the minus p dV term a plus F dL term. If you pull on the string to make it an amount dL longer, then an amount of F dL of work is ...
A detailed solution is given.