Suga has faced criticism over a coronavirus response seen as too slow and for holding Olympics