Technically, B is agreeing with A by saying 'Yes'. However, in everyday language-usage we just don't normally confirm a negative by using 'Yes'. For example, people do not say: "Yes, I won't," do they? Most of us, therefore, would interpret the 'Yes' here as meaning: 'Yes, I am going to do this.'