On Fri, Jan 25, 2013 at 4:13 AM, IZ3GME Marco iz3gme.marco@gmail.comwrote:
It's a rounding problem due to the internal representation of float.
I tried the following version of convert_freq_to_bytes which seems to works but may be Dan or some other python expert have a better solution.
def convert_freq_to_bytes(freq): bytes = [ 0 for x in range(0,8) ] # init list with 8 times 0 real_freq = "%3.5f" % freq # now is a string real_freq = int(real_freq[0:3]+ real_freq[4:]) # and now an int for i in range(7, -1, -1): # go from 7 to 0 bytes[i] = real_freq%10 # extract last digit real_freq /= 10 # throw away last digit return bytes
73 de IZ3GME Marco
I still get 146.63999 out for 146.64000 in.
In settings.py, I had changed 'precision=4' to 'precision=5'
class RadioSettingValueFloat(RadioSettingValue): """A floating-point setting""" def __init__(self, minval, maxval, current, resolution=0.001, precision=4):
The reason I did this was to get 5 places past the decimal. Maybe that was the wrong way to accomplish that. Anyway, I changed back to 'precision=4' and now the result is 146.6400, which is better than 146.63999, but still not exactly what I am looking for.
Thanks again. You help and guidance is greatly appreciated. Jim KC9HI