He told the Heraldthat, as well as rebuilding his party's support, he wanted to ensure his member's bill on End of Life Choice could progress, as "assisted dying is the kind of issue that brought me into politics".
If he became a minister the voluntary euthanasia bill would have to be scrapped, and he was not confident any other MPs would put forward similar legislation.
Mr Seymour's bill has not been drawn yet. The ballot could be cruel, he said, but it would be unlucky for the bill not to come up over the coming year.
The ministerial offer was put to Mr Seymour on Sunday morning, but he said it had been well-signalled for several months.
He consulted widely on what he should do, including with Act's board, his Epsom electorate committee, key staff and supporters, and friends.
"I am sort of getting pulled in four or five different directions, and on balance I think staying where I am is the right place to be."