He had a wife named Lorna, who he treated very poorly. Ashton did not know that because of his behavior, Lorna became attracted to, and fell in love with, his parter Giles.
Ashton found out about the affair, probaby by finding a letter hidden beneath Lorna's pillow. He also found out about the job that Lord Holanthrus had open. Lorna had planned to divorce Ashton, and run away with Giles, and use the new job as a cover. Giles was also planning to use Ashtons promotion to head libraian as a consolation for losing his wife. With this information, Ashton murdered both Giles and Lorna, and hid their bodies in a secret compartment in the library.
Ashton left Lord Gervasius, saying Giles and Lorna had ran off together, and that the library, 'held too many painful memories', and went for the new job with Lord Holanthrus.