The decision must be yours but I do understand your reservations and those of your fellow travellers. I have never been to Brazil but I have spent quite a few years in malarial countries/areas. I took anti-malarial medication for the first few weeks but then stopped (unpleasant plus unknown long term effects - as you say, only reduced risk of infection) and my wife and young children did not take any either once they joined me. None of us ever got malaria. The European medical profession (UK included) will no doubt strongly advise anyone going to any of these places to take the medication - that is what they must do for fear of being criticised.
To risk getting malaria one must be bitten by an infected mosquito. What my family and I did was to wear long sleeves and trousers when (rarely) out and about between dusk and dawn (including eating out, etc.), together with a good mosquito repellent (Mosquito Milk it was called, no longer available I believe, Avon's Skin So Soft I think it is called that is said to be very effective) applied on the forehead, cheeks, behind ears, all around the neck and at wrists and ankles plus a strip down the trouser legs for good measure.
We used a good insecticide (Baygon is the best) in the accommodation - a continuous strip of spray all around windows and doors, into ceiling corners, and under beds and other furniture (every evening without fail if in other places than our own home) and kept windows and doors closed. If you have access to a good bed net then use it.