Your best bet for a large brown will be in the Little River, probably above Metcalf Bottoms. But there is a reason that these wild browns grow above 10-12"...they're very elusive. 15"+ Browns do exist and are caught from time to time, but usually by folks that have been stalking them for awhile. If you are coming to try and catch a big brown, you might be disappointed.
On the other hand, if you fish on the LR above Elkmont, or as Gerry suggested on Lynn Camp Prong, you can combine a nice (not too strenuous) hike with some really good wild trout fishing (smaller but more plentiful and feistier than large stockers).
If you want a more strenuous hike and fish expedition, you could travel up the Chimney Tops trail to the 3rd or 4th bridge and fish Road Prong and let the women-folk hike up to the top of the Chimney Tops. It'll be more work, but the effort could pay greater dividends, for both groups (Brookies and better views), depending on the time of year you are heading over...