You can. Create a new playlist showing only album and artist. You will still have all the tracks shown, you can't get rid of these yet. Select all the albums you want to list and add to your playlist, you will now have track, album and artist. Go to edit, select all, copy.
Open Excel and paste. You now have the three columns listed. Delete the Track column.
You now have lots of duplicate album/artists. In excel select all in your new list, go to the Data menu and select Filter/Advanced Filter. Under Action, click filter the list in-place, at the bottom check the Unique Records Only checkbox and click OK. This should remove pretty much all your duplicates leaving only single rows for album artist (assuming your albums all have a single artist).
You might get an error message when using advanced filter that says it cannot find column headings. Ignore.