Currently they have failed by default. Note also the environment variable MYSQL_TEST_SKIP_CONNECT_FAILURE to control the default (skip or fail)