James and Agnes Taylor were both born to aristocracy; but the oldest children in their families inherited the money, titles and property. Since they were younger children, they had to earn their living, so even though James was well-educated and came from wealth, he had to work. He found a job with the British government, taxing manufactured goods.
The small town of Milnthorpe invited shoppers to market every Friday. The farmers came to town just before dawn, carrying their goods in small carts. Some travelled along the Kent River which meandered through the fields on its way to Morcambe Bay. Others hurried to town past historical castles in the area that crumbled, struggling to survive, some against ancient attacks of the Scots to the north and others from decay and lack of use.