For now, you can just split it if you don't want to lose your logs
$ split -b10m /var/log/btmp btmp_
will split the file into 10-megabyte parts, named btmp_aa, btmp_ab, btmp_ac ... btmp_zy, btmp_zz
You can also write -l120 instead of -b10m to split a file into 120-line parts
The original file remains intact, so just do a 'mv btmp_xx btmp', where xx is last created suffix.
For more information, read split manual or infodoc (info coreutils 'split invocation')