I think you should solve it this way:
Write ax/(1+bx) as ax(1+bx)^-1 and expand as
ax(1-bx+b^2x^2+terms in x^3 or above)
so exp(ax(1-bx+b^2x^2+terms in x^3 or above)) is:
1+ax(1-bx+b^2x^2+terms in x^3 or above)+a^2x^2(1-bx+b^2x^2+terms in x^3 or above)^2
=1+ax-abx^2+a^2x^2+terms in x^3 or above.
If the x terms and x^2 terms in this series must be the same as those of
(1+x)^n=1+nx+n(n-1)x^2+terms in x^3 or above, then
a=n and n^2-nb=n(n-1)/2, so b=(n+1)/2 as long as n is not equal to 0.
You can see that this works for n=1 giving a=1 and b=1
and for n=2 giving a=2 and b=3/2