Horncastle in Lincs has a nice hotel called Admiral Romney (or Rodney?!). More at the top end of Lincs (?), Louth has a great market and is a nice town anyway, some lovely shops.
Markets too, Horncastle (Thurs and Sats, Farmers version 2nd Thurs), Louth = Weds, Fris, Sats, Farmers version 2nd Friday, 4th Weds). Take a cool box?!
Beware that crossing Norfolk takes a while, one of very few counties with no motorway, and the road from Peterborough to Norwich gets busy and seems endless?!