TO_CHAR(payment_date, 'Day') returns a string padded with spaces ('Monday ').

To suppress the spaces, use the FM modifier ("fill mode")

SELECT COUNT(*)
FROM payment
WHERE (TO_CHAR(payment_date, 'FMDay')) = 'Monday'

alternatively be explicit and use trim()

SELECT COUNT(*)
FROM payment
WHERE (trim(TO_CHAR(payment_date, 'Day'))) = 'Monday'

However, I would recommend to not use locale specific values (on my computer the above would always return 0 as I have a different language setting).

Using numbers e.g. with extract(isodow from ..) is much more reliable.

Use dow(day of week).
The day of the week (0 - 6; Sunday is 0) (for timestamp values only)

SELECT COUNT(*)
FROM payment
WHERE EXTRACT(DOW FROM payment_date)= 1;

For more details on the dow check out the documentation.

SELECT COUNT(*) AS day_of_payment 
FROM payment
WHERE TO_CHAR (payment_date,'DAY') = 'MONDAY   '

Considering that "blank padded to 9 characters", which means instead of returning 'Monday' it returns 'Monday ' with extra spaces to fill up at least 9 spaces.