Set up the following view in Media Library – Music – Views:

#,Group by,Function,Mode,Sort by,Sort type,Sort Ascending
1 Artist, index, index, , ,true
2 Artist, , , , ,true
3 Album, ,images, , ,true
4 , ,album-tracks,Track,int,true

How it looks like in the configuration tool

As you can see here, the index automatically selects the letters that is necessary for your collection. This screen shot is from my laptop, where I have a very limited collection…