[chirp_devel] RFC settings import export
Hi all from recent comments in users list I'm thinking to implement import-export of settings to cvs. I'm not going to try any cross model support at first.
Comments? Do you see any drawback?
73 de IZ3GME Marco
In my limited dealings with settings on a few radios, they seem very proprietary. How can you "serialize" these? Noting that not all radios have settings stored within a single array or struct - there may be multiple places, and then some custom logic on how it's read from and stored to radio image.
Now that I think about it, perhaps you could export just Chirp's view of the settings, as it is more or less a tree structure, with a few fixed types and values. Seems like it would work ok to import settings between same models, but would get very tricky (if not impossible) to implement across models.
-J
________________________________ From: IZ3GME Marco iz3gme.marco@gmail.com To: chirp_devel@intrepid.danplanet.com Sent: Sunday, December 29, 2013 6:05 PM Subject: [chirp_devel] RFC settings import export
Hi all from recent comments in users list I'm thinking to implement import-export of settings to cvs. I'm not going to try any cross model support at first.
Comments? Do you see any drawback?
73 de IZ3GME Marco _______________________________________________ chirp_devel mailing list chirp_devel@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_devel Developer docs: http://chirp.danplanet.com/projects/chirp/wiki/Developers
Also, I seem to recall some sort of long description field in settings that I attempted to use, but did not do anything. I was looking for some UI control akin to a label or tooltip (i.e., to provide a longer description or notes about a particular setting) but could not find anything that did that. If such a thing existed, it would be useful in an export
On the format of csv, seems like the following fields would be useful: setting_name, group_name, setting_description, type, value
types: boolean, int, float, list, string
Not sure if having a column for the possible values of a setting, such as range or list, would be of any value to the user.
Seems like there would need to be some logic at import time to resolve any conflicts or missing, as I dont think there is a concept of "default values" for settings within chirp at this point. Along these lines, settings can also change/expand over time with newer versions of software. Conflict/missing resolution would also be needed here. I guess if it's importing settings over an existing radio image, this may be easier. But there is still the general idea of how to deal with InvalidValueError exceptions!
-J
________________________________ From: Jens J. kd4tjx@yahoo.com To: "chirp_devel@intrepid.danplanet.com" chirp_devel@intrepid.danplanet.com Sent: Sunday, December 29, 2013 6:11 PM Subject: Re: [chirp_devel] RFC settings import export
In my limited dealings with settings on a few radios, they seem very proprietary. How can you "serialize" these? Noting that not all radios have settings stored within a single array or struct - there may be multiple places, and then some custom logic on how it's read from and stored to radio image.
Now that I think about it, perhaps you could export just Chirp's view of the settings, as it is more or less a tree structure, with a few fixed types and values. Seems like it would work ok to import settings between same models, but would get very tricky (if not impossible) to implement across models.
-J
________________________________ From: IZ3GME Marco iz3gme.marco@gmail.com To: chirp_devel@intrepid.danplanet.com Sent: Sunday, December 29, 2013 6:05 PM Subject: [chirp_devel] RFC settings import export
Hi all from recent comments in users list I'm thinking to implement import-export of settings to cvs. I'm not going to try any cross model support at first.
Comments? Do you see any drawback?
73 de IZ3GME Marco _______________________________________________ chirp_devel mailing list chirp_devel@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_devel Developer docs: http://chirp.danplanet.com/projects/chirp/wiki/Developers
_______________________________________________ chirp_devel mailing list chirp_devel@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_devel Developer docs: http://chirp.danplanet.com/projects/chirp/wiki/Developers
On Dec 29, 2013 4:05 PM, "IZ3GME Marco" iz3gme.marco@gmail.com wrote:
Hi all from recent comments in users list I'm thinking to implement import-export of settings to cvs. I'm not going to try any cross model support at first.
Comments? Do you see any drawback?
I don't see how this is possible with CSV. CSV uses the first line to indicate keys and positions of columns throughout the file. Then the rest of the file is one line per memory. I don't know how you will encode settings.
It makes more sense to me to begin by supporting Import/Export of settings via img from identical radio models. For example, import uv5r.img into another uv5r. For this you will not need any CSV encoding or translation between models. I think this feature would satisfy most user's need for settings import.
Tom KD7LXL
On 12/30/2013 1:03 AM, Tom Hayward wrote:
It makes more sense to me to begin by supporting Import/Export of settings via img from identical radio models. For example, import uv5r.img into another uv5r. For this you will not need any CSV encoding or translation between models. I think this feature would satisfy most user's need for settings import.
Couldn't that use case of moving settings to an identical radio already be handled by just using the source image after importing the frequencies from the destination radio's image?
The recently requested use case was the possibility of improved accessibility for the visually impaired by allowing CSV export and import. As mentioned, this would need 100% fidelity in the export and re-import to be useful, so would be difficult to achieve, especially with CSV.
The use case that I haven't seen mentioned, is exporting to some textual representation to facilitate settings comparisons. This is useful for not only comparing two radios, but also for determining what settings have changed over time on a single radio. I use G4HFQ's FTBVX8 software in that manner. The settings user interface is table oriented and can be exported to text files making compares possible, even on radios with a lot of settings like the VX-8.
As this can be very useful for diagnostic purposes, I'd be interested in seeing some sort of textual export/report, even if there were no import, initially just an additional developer function. More of the settings meta data would need to be exposed, which would wind up providing additional documentation for each radio's settings. I'm guessing that is where Jens J. is headed recently asking about doc strings for settings.
Just adding my thoughts, the subject did say this was a request for comments, right?
A happy and healthy New Year to all, --Rob
Im not sure if you are getting at this, but one "dirty little secret" of importing images into other "identical" radios is that, in at least some models, there may be other non-exposed settings (e.g., power calibrations, deviations, level settings, etc) that are stored in the radio image, and are (or should be) specific to exactly that individual radio. While they _might_ be mostly portable between same model of radios, there could be potential issues.
The best approach should be to import export channels and settings directly from existing image, rather than opening the image and uploading it directly to the radio. This is up to the user to do this, but currently I doubt most users understand this, nor is it really documented anywhere (that Im aware of).
The "ability to import settings from existing images of same model of radio" should be the new functionality here. I think Tom mentioned this and I vote for this approach.
-J
________________________________ From: Robert Terzi rct@r-t.org To: chirp_devel@intrepid.danplanet.com Sent: Monday, December 30, 2013 11:36 AM Subject: Re: [chirp_devel] RFC settings import export
On 12/30/2013 1:03 AM, Tom Hayward wrote:
It makes more sense to me to begin by supporting Import/Export of settings via img from identical radio models. For example, import uv5r.img into another uv5r. For this you will not need any CSV encoding or translation between models. I think this feature would satisfy most user's need for settings import.
Couldn't that use case of moving settings to an identical radio already be handled by just using the source image after importing the frequencies from the destination radio's image?
The recently requested use case was the possibility of improved accessibility for the visually impaired by allowing CSV export and import. As mentioned, this would need 100% fidelity in the export and re-import to be useful, so would be difficult to achieve, especially with CSV.
The use case that I haven't seen mentioned, is exporting to some textual representation to facilitate settings comparisons. This is useful for not only comparing two radios, but also for determining what settings have changed over time on a single radio. I use G4HFQ's FTBVX8 software in that manner. The settings user interface is table oriented and can be exported to text files making compares possible, even on radios with a lot of settings like the VX-8.
As this can be very useful for diagnostic purposes, I'd be interested in seeing some sort of textual export/report, even if there were no import, initially just an additional developer function. More of the settings meta data would need to be exposed, which would wind up providing additional documentation for each radio's settings. I'm guessing that is where Jens J. is headed recently asking about doc strings for settings.
Just adding my thoughts, the subject did say this was a request for comments, right?
A happy and healthy New Year to all, --Rob
_______________________________________________ chirp_devel mailing list chirp_devel@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_devel Developer docs: http://chirp.danplanet.com/projects/chirp/wiki/Developers
On 12/30/2013 12:57 PM, Jens J. wrote:
Im not sure if you are getting at this, but one "dirty little secret" of importing images into other "identical" radios is that, in at least some models, there may be other non-exposed settings (e.g., power calibrations, deviations, level settings, etc) that are stored in the radio image,
Ok that's a bit disturbing, I wasn't aware of that. What models do that?
The clone mode radios I have played with, Yaesus and Wouxuns, appear to keep the calibration data outside of the region that is cloned or is write protected. (Cloning functionality is built into those radios, so it makes sense that the manufacturer's would figure out to protect radio specific data.)
The best approach should be to import export channels and settings directly from existing image, rather than opening the image and uploading it directly to the radio. This is up to the user to do this, but currently I doubt most users understand this, nor is it really documented anywhere (that Im aware of).
So should there be warnings on radios that aren't "clone safe"?
However this still brings back the point of how do you know what you are importing and why? How do you compare two radios? You can try to flip back and forth between open tabs, or take screen shots. For simple radios that is probably fine, but increasingly tedious on more feature-rich radios.
The "ability to import settings from existing images of same model of radio" should be the new functionality here. I think Tom mentioned this and I vote for this approach.
To prevent copying unwanted data hiding in unused fields, I'm assuming the only settings that will get imported/export are those that have defined setting interfaces. Additionally, I'm guessing you might want to have the changes go through the setting interface so any error checking will fire. So really I'm just suggesting (or hoping for) functionality that walks the known settings and can dump them in some sort of textual representation that can be examined outside of the chirp gui.
--Rob
The "ability to import settings from existing images of same model of radio" should be the new functionality here. I think Tom mentioned this and I vote for this approach.
To prevent copying unwanted data hiding in unused fields, I'm assuming the only settings that will get imported/export are those that have defined setting interfaces. Additionally, I'm guessing you might want to have the changes go through the setting interface so any error checking will fire. So really I'm just suggesting (or hoping for) functionality that walks the known settings and can dump them in some sort of textual representation that can be examined outside of the chirp gui.
Robert, that's my idea ... you surely described it better than me. I'll implement export/import first and only after, if we still see the need, i'll make the direct import from images.
I forget to tell it before but, please, "chirp people" accept my best wishes for the upcoming new year.
73 de IZ3GME Marco
On Mon, Dec 30, 2013 at 9:36 AM, Robert Terzi rct@r-t.org wrote:
On 12/30/2013 1:03 AM, Tom Hayward wrote:
It makes more sense to me to begin by supporting Import/Export of settings via img from identical radio models. For example, import uv5r.img into another uv5r. For this you will not need any CSV encoding or translation between models. I think this feature would satisfy most user's need for settings import.
Couldn't that use case of moving settings to an identical radio already be handled by just using the source image after importing the frequencies from the destination radio's image?
As Jens mentioned, there's other stuff in the radio img you don't want to blindly copy, like calibration data. I was suggesting copying only the known settings bytes. Since it's the same model, you can simply copy the settings blob without any conversion. The UI for this feature could simply be a checkbox under the frequency import table that says "also import settings?".
The recently requested use case was the possibility of improved accessibility for the visually impaired by allowing CSV export and import. As mentioned, this would need 100% fidelity in the export and re-import to be useful, so would be difficult to achieve, especially with CSV.
I have yet to understand this use case, so I was ignoring it. Can anyone explain how putting settings in a CSV improves accessibility? Guessing here, but is LibreOffice spreadsheet-to-speech really that much better?
I don't see how CSV is going to be a reasonable or effective way of encoding settings information.
The use case that I haven't seen mentioned, is exporting to some textual representation to facilitate settings comparisons. This is useful for not only comparing two radios, but also for determining what settings have changed over time on a single radio. I use G4HFQ's FTBVX8 software in that manner. The settings user interface is table oriented and can be exported to text files making compares possible, even on radios with a lot of settings like the VX-8.
I've never seen this. I'm curious how settings are put in a table format. Key-value seems more natural.
As this can be very useful for diagnostic purposes, I'd be interested in seeing some sort of textual export/report, even if there were no import, initially just an additional developer function. More of the settings meta data would need to be exposed, which would wind up providing additional documentation for each radio's settings. I'm guessing that is where Jens J. is headed recently asking about doc strings for settings.
It probably wouldn't be too involved to add a JSON output to RadioSettingGroup. You could run a diff on this. This makes some sense to me for power users, but not for accessibility.
Tom KD7LXL
On 12/30/2013 1:10 PM, Tom Hayward wrote:
It probably wouldn't be too involved to add a JSON output to RadioSettingGroup. You could run a diff on this. This makes some sense to me for power users, but not for accessibility.
To me, that would be cool and would address my needs even if it was only available after turning on the developer functions.
I've never seen this. I'm curious how settings are put in a table format. Key-value seems more natural.
Agreed, but since it fits the tool many people have available to them, I'm seen a bunch of software that outputs key value pairs as a 2-3 column table. To me I'm happy with any text that I can take a pass at with the text processing tools at my disposal, so JSON, tab separated, csv all good.
As Jens mentioned, there's other stuff in the radio img you don't want to blindly copy, like calibration data. I was suggesting copying only the known settings bytes. Since it's the same model, you can simply copy the settings blob without any conversion. The UI for this feature could simply be a checkbox under the frequency import table that says "also import settings?".
That sounds pretty reasonable.
Just curious, are radio's like the baofeng with different firmware versions considered the same model or different?
Hi Tom ,Jens and Robert thanks for your thoughts :) my idea is to export RadioSetting list to a separate csv from memories one, the format can be:
setting key, setting description, field type, acceptable values, value
I will probably use tab instead of comma as field separator to avoid problems with string settings that allow commas in value. At import we can use only key and value to update each RadioSetting in list if the key exists and let the set_value method do the usual checks, at end we can show a dialog to let user know if there were any unknown key or unacceptable value.
This will obviously export/import only well known settings we already support using existing code do same validation done at user input. Being basically a text file the csv (i suppose) can be easily read by text2speech and edited with non visual editors. Someone can also remove some rows from the file to import only some of the settings and use it as a sort of "stock settings" (I think to people managing a fleet of radio or avoid importing any calibration setting from different radio).
I also like Tom's idea to import settings directly from img but this have in my opinion some limitation so i think we can go providing both cvs and img settings import.
Although you have expressed some (stimulating) criticism I dont see in this discussion any concrete reason for not try implementing it. I'll translate my idea in a patch in next days so you all can try it and probably better understand than from my english ;)
73 de IZ3GME Marco
participants (4)
-
IZ3GME Marco
-
Jens J.
-
Robert Terzi
-
Tom Hayward