Actually it’s a bit more general than just mapfile:
$ declare -n ref=XXX[0]; declare -A ref; ref[foo]=bar
$ declare -p XXX[0]
declare -A XXX[0]=([foo]="bar" )
$ declare -n ref=XXX[0]; ref+=([2]=x)
$ declare -p XXX[0]
declare -a XXX[0]=([2]="x")
$ declare -n ref=XXX[0]; read -a ref <<< "A B C"
$ declare -p XXX[0]
declare -a XXX[0]=([0]="A" [1]="B" [2]="C")
OTOH coproc now handles this a bit better, in that it won’t create the array, though it will use one if one exists.
$ declare -n ref=XXX[0]; unset ref; coproc ref { :; }; declare -p ${!ref}
bash: `XXX[0]': not a valid identifier
bash: declare: XXX[0]: not found
$ declare -n ref=XXX[0]; unset ref; ref=(); coproc ref { :; }; declare -p ${!ref}
declare -a XXX[0]=([0]="61" [1]="56")