[chirp_users] Chirp won't read CSV files correctly
I have a KG-UV8D. I can use Chirp to read from the radio, and save the image and create a CSV file. But if I modify the CSV file and then try to open it from Chirp, I get various errors. I was able to successfully import 15 memories (plus the default 0 channel the Chirp creates -- don't know what this is), but when Chirp encounters the next line (16th entry) in my CSV file if barfs and says "Mismatched column numbers line 17".
I can delete the first 15 rows in my CSV file and create a new series of entries with the second half of my memory list, starting with Channel 16, and it works, until it gets to the 17th line, then issues the error "Mismatched column numbers line 17".
Now, I'm certain that import/export has to be a very basic function so I must be doing something wrong any help out there?
Chirp version (the only one I can find on the web site) : chirp-daily-20160224-installer.exe Windows 7 Microsoft Excel 2003
I also created an account on your the web site, but cannot login -- I get the message login info invalid-- do I have to wait for an email after creating an account?
Chris Nordstrom
On Fri, Feb 26, 2016 at 11:04 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
I have a KG-UV8D. I can use Chirp to read from the radio, and save the image and create a CSV file. But if I modify the CSV file and then try to open it from Chirp, I get various errors. I was able to successfully import 15 memories (plus the default 0 channel the Chirp creates -- don't know what this is), but when Chirp encounters the next line (16th entry) in my CSV file if barfs and says "Mismatched column numbers line 17".
Computers don't lie--I bet line 17 of your CSV file has mismatched line numbers, just as the error says. If you attach or post a link to your CSV file, I could tell for sure.
Excel has a known bug where it will remove empty cells from a CSV file. Maybe that's what did it. I generally don't recommend using anything other than Chirp to edit Chirp files.
More background reading here: http://chirp.danplanet.com/projects/chirp/wiki/CSV_HowTo
Tom KD7LXL
This is the error I get when importing or opening my CSV file:
I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time.
So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one:
The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out.
I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations.
Chris Nordstrom
I assume the file you attached was the one that gave you the second (smaller) set of errors.
If you look at it in Excel, you'll see that rows 33, 35 and 40 are the only rows that have nothing in the rToneFreq and cToneFreq columns.
There are two other rows that have nothing in the Tone column (rows 2 and 3), but these still have values in the other two columns. Looks like you have to have a value of some sort in these two columns.
Could you please post the file that gave you the errors from row 17 on, so we can have a look at it too.
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Saturday, February 27, 2016 4:15 PM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
This is the error I get when importing or opening my CSV file:
I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time.
So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one:
The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out.
I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations.
Chris Nordstrom
------------------------------------------------------------------------------
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11703 - Release Date: 02/26/16
Dave,
Yup, success! When I put 88.5 in the */rToneFreq/* column (also used 67, I guess any valid tone freq) _/and/_ deleted column headers from any column that contains no data _/and/_ entered the correct number of rows in *Memory Range*---badda bing -- it works.
I also figured out that I cannot directly /*Upload*/ a CSV file to the radio, it has to be an IMG file, which makes sense, but the method for getting an IMG file is not obvious and with no documentation (that I could find), eew! So what I did was */Download/* from the radio, delete all except the first row, save it as an "Empty" IMG file. Next, I */Open/* that IMG file then /*Import*/ the CSV file containing the memory channel data I want, and magically, it becomes an IMG file that I can now /*Upload*/ to the radio. Is this how it is supposed to work or is there another way to do this? Does it work basically the same regardless of the radio brand?
Attached is the CSV file that causes the Column number mismatch error. The only difference between this file and the one that successfully imports is that there*IS* column heading text in columns M, N, O, P and Q, but there is no data in those columns. When I removed the heading text from just those columns, the problem goes away.
Thanks for your encouragement and ideas,
Regards, Chris W7CFN
P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me.
On 2/26/2016 10:43 PM, Dave VK2FDJS wrote:
I assume the file you attached was the one that gave you the second (smaller) set of errors. If you look at it in Excel, you'll see that rows 33, 35 and 40 are the only rows that have nothing in the rToneFreq and cToneFreq columns. There are two other rows that have nothing in the Tone column (rows 2 and 3), but these still have values in the other two columns. Looks like you have to have a value of some sort in these two columns. Could you please post the file that gave you the errors from row 17 on, so we can have a look at it too.
Regards, Dave VK2FDJS
----- Original Message ----- *From:* Chris Nordstrom <mailto:cfnordstrom@gmail.com> *To:* chirp_users@intrepid.danplanet.com <mailto:chirp_users@intrepid.danplanet.com> *Sent:* Saturday, February 27, 2016 4:15 PM *Subject:* Re: [chirp_users] Chirp won't read CSV files correctly This is the error I get when importing or opening my CSV file: I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time. So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one: The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out. I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations. Chris Nordstrom ------------------------------------------------------------------------ _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com ------------------------------------------------------------------------ No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2015.0.6189 / Virus Database: 4537/11703 - Release Date: 02/26/16
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
On Sat, Feb 27, 2016 at 10:17 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me.
Chris,
I'm puzzled here because I tried to reply with a link to the documentation a mere 20 minutes after your original post. In fact, that reply of mine even appears in the archive: http://intrepid.danplanet.com/pipermail/chirp_users/2016-February/009383.htm... I apoligize if I somehow messed up the email such that it was not mailed to you and the others. Here is the snippet again with a link to the documentation, explicitly detailing the procedure you couldn't find documentation for:
On Fri, Feb 26, 2016 at 11:25 AM, Tom Hayward tom@tomh.us wrote:
More background reading here: http://chirp.danplanet.com/projects/chirp/wiki/CSV_HowTo
Scroll to "Importing the CSV file into a radio" for the procedure.
If you would like to help improve the documentation, just create an account on that site (as you would to report an issue) and ask Dan to give you wiki editor access.
Tom KD7LXL
Thanks Tom, sorry, I did see this basic documentation, but did not make the connection between these procedures and the errors I was getting related to column numbers and row conversion. I guess it does answer my initial question about how to get a CSV file into an IMG file for upload to the radio.
But I could not find where it says you should delete column header text for columns that contain no data and it also does not say that the Memory Range entry on the edit window must match the number of rows you are importing; the error you get doesn't lead you make this connection. Did I miss this too?
The software is not particularly intuitive, but I guess it has to have some limitations so that it can work with the most number of radios. However, I've figured it all out for my KG-UV8D and it works great.
Regards, Chris W7CFN
On 2/27/2016 10:25 AM, Tom Hayward wrote:
On Sat, Feb 27, 2016 at 10:17 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me.
Chris,
I'm puzzled here because I tried to reply with a link to the documentation a mere 20 minutes after your original post. In fact, that reply of mine even appears in the archive: http://intrepid.danplanet.com/pipermail/chirp_users/2016-February/009383.htm... I apoligize if I somehow messed up the email such that it was not mailed to you and the others. Here is the snippet again with a link to the documentation, explicitly detailing the procedure you couldn't find documentation for:
On Fri, Feb 26, 2016 at 11:25 AM, Tom Hayward tom@tomh.us wrote:
More background reading here: http://chirp.danplanet.com/projects/chirp/wiki/CSV_HowTo
Scroll to "Importing the CSV file into a radio" for the procedure.
If you would like to help improve the documentation, just create an account on that site (as you would to report an issue) and ask Dan to give you wiki editor access.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
On Sat, Feb 27, 2016 at 11:33 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
But I could not find where it says you should delete column header text for columns that contain no data and it also does not say that the Memory Range entry on the edit window must match the number of rows you are importing; the error you get doesn't lead you make this connection. Did I miss this too?
Not sure what you mean about memory range view being related to the number of rows imported. I've imported outside the currently selected range many times and never had an issue. I just did it again now to verify. The memory range view is just a view; it doesn't affect imports.
Regarding the Excel bugs, there's only so much we can do about people using 3rd party software to edit Chirp files and getting unexpected results. As I said in my first email, I recommend editing Chirp files with Chirp. When you edit with Chirp, it maintains all the proper data in the proper columns. There is no need to delete headers or fill your own defaults--it's all automatic in Chirp. Feel free to write some documentation about the caveats you discovered when editing Chirp files in Excel.
Tom KD7LXL
Tom,
Okay, I just tested it again and it works as you say. When I was getting the import errors before--caused by not having data in key fields--it appeared that it stopped at the default number--25--which was only limiting what I could see, not what was imported. So it works fine.
I am not aware of these bugs in Excel that were mentioned. After creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in /*rToneFreq*/? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Regards, Chris W7CFN
On 2/27/2016 11:57 AM, Tom Hayward wrote:
On Sat, Feb 27, 2016 at 11:33 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
But I could not find where it says you should delete column header text for columns that contain no data and it also does not say that the Memory Range entry on the edit window must match the number of rows you are importing; the error you get doesn't lead you make this connection. Did I miss this too?
Not sure what you mean about memory range view being related to the number of rows imported. I've imported outside the currently selected range many times and never had an issue. I just did it again now to verify. The memory range view is just a view; it doesn't affect imports.
Regarding the Excel bugs, there's only so much we can do about people using 3rd party software to edit Chirp files and getting unexpected results. As I said in my first email, I recommend editing Chirp files with Chirp. When you edit with Chirp, it maintains all the proper data in the proper columns. There is no need to delete headers or fill your own defaults--it's all automatic in Chirp. Feel free to write some documentation about the caveats you discovered when editing Chirp files in Excel.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
On Feb 27, 2016 13:48, "Chris Nordstrom" cfnordstrom@gmail.com wrote:
I am not aware of these bugs in Excel that were mentioned. After
creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in rToneFreq? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Yes, you need to provide appropriate defaults if you're not editing with Chirp (when you edit a CSV with Chirp, it will fill in all the defaults for you). Here's a quote from the documentation I linked earlier:
"Field values must be in the original format and one of the accepted values for the column type."
Excel is a generic spreadsheet program. If you want to use it to edit Chirp CSV files, you will need to handle the data validation yourself to ensure it stays in a format compatible with Chirp.
Tom KD7LXL
I am having trouble trying to load my Yeasu FT 8800, I get a error message each time. I downloaded the drivers for the cable and am using a MacBook Pro. Any suggestions?
Kevin Karzin 661-810-4249 www.kevinkarzinphotography.com President - Santa Clarita Valley Photographers Association Member - Press Photographers Association of Greater Los Angeles Member - Southern California Photographers Association
On Feb 27, 2016, at 2:31 PM, Tom Hayward tom@tomh.us wrote:
On Feb 27, 2016 13:48, "Chris Nordstrom" cfnordstrom@gmail.com wrote:
I am not aware of these bugs in Excel that were mentioned. After creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in rToneFreq? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Yes, you need to provide appropriate defaults if you're not editing with Chirp (when you edit a CSV with Chirp, it will fill in all the defaults for you). Here's a quote from the documentation I linked earlier:
"Field values must be in the original format and one of the accepted values for the column type."
Excel is a generic spreadsheet program. If you want to use it to edit Chirp CSV files, you will need to handle the data validation yourself to ensure it stays in a format compatible with Chirp.
Tom KD7LXL
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Kevin Karzin at kkarzin@aol.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
Did you READ the radio before trying to write to the radio?
Jardy Dawson WA7JRD Ham Radio
Message relayed through SETI (Search for Extra-Terrestrial Intelligence, although no terrestrial intelligence has been found yet, either)
On Feb 27, 2016, at 14:36, Kevin Karzin via chirp_users chirp_users@intrepid.danplanet.com wrote:
I am having trouble trying to load my Yeasu FT 8800, I get a error message each time. I downloaded the drivers for the cable and am using a MacBook Pro. Any suggestions?
Kevin Karzin 661-810-4249 www.kevinkarzinphotography.com President - Santa Clarita Valley Photographers Association Member - Press Photographers Association of Greater Los Angeles Member - Southern California Photographers Association
On Feb 27, 2016, at 2:31 PM, Tom Hayward tom@tomh.us wrote:
On Feb 27, 2016 13:48, "Chris Nordstrom" cfnordstrom@gmail.com wrote:
I am not aware of these bugs in Excel that were mentioned. After creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in rToneFreq? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Yes, you need to provide appropriate defaults if you're not editing with Chirp (when you edit a CSV with Chirp, it will fill in all the defaults for you). Here's a quote from the documentation I linked earlier:
"Field values must be in the original format and one of the accepted values for the column type."
Excel is a generic spreadsheet program. If you want to use it to edit Chirp CSV files, you will need to handle the data validation yourself to ensure it stays in a format compatible with Chirp.
Tom KD7LXL
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Kevin Karzin at kkarzin@aol.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Jardy at jardy72@yahoo.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
Right, well as I've said, when you create generic CSV file from Chirp and go look at it, it still contains all the empty fields and the extra commas at the end of the row. So....
Regards, Chris W7CFN
On 2/27/2016 2:31 PM, Tom Hayward wrote:
On Feb 27, 2016 13:48, "Chris Nordstrom" <cfnordstrom@gmail.com mailto:cfnordstrom@gmail.com> wrote:
I am not aware of these bugs in Excel that were mentioned. After
creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in rToneFreq? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Yes, you need to provide appropriate defaults if you're not editing with Chirp (when you edit a CSV with Chirp, it will fill in all the defaults for you). Here's a quote from the documentation I linked earlier:
"Field values must be in the original format and one of the accepted values for the column type."
Excel is a generic spreadsheet program. If you want to use it to edit Chirp CSV files, you will need to handle the data validation yourself to ensure it stays in a format compatible with Chirp.
Tom KD7LXL
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
On Sat, Feb 27, 2016 at 7:52 PM, Chris Nordstrom cfnordstrom@gmail.com wrote:
Right, well as I've said, when you create generic CSV file from Chirp and go look at it, it still contains all the empty fields and the extra commas at the end of the row. So....
The "extra" commas are a critical, required part of the Chirp CSV format.
I'm having a hard time understanding why you need Excel in the first place. It sounds like you are a relatively new user of Chirp, yet using third party software to edit Chirp files is arguably one of the most advanced things you can do with Chirp. As you've found, it requires understanding the specific details of the file formats and any errors that might be introduced by specific third party editors. A lot of effort has been put into the built-in Chirp editor. It supports copy, paste, inserting rows, removing rows, populating safe default values, automatic repeater offsets for various band plans around the world, bulk row editing (i.e., setting power level to high on 20 channels at once). What is missing from Chirp that drove you to an external editor? I'm not implying there are no valid uses of external editors--indeed, this is one of the benefits of using CSV encoding--but most of the time it just makes things more difficult.
Tom KD7LXL
On Feb 27, 2016 11:33 PM, "Tom Hayward" tom@tomh.us wrote:
I'm having a hard time understanding why you need Excel in the first place. It sounds like you are a relatively new user of Chirp, yet using third party software to edit Chirp files is arguably one of the most advanced things you can do with Chirp.
I am a fairly advanced user of Chirp myself at this point, but to me it is easy to understand how a new user gets stuck in this mess because I've done it myself as a new user often enough. If the point of using the CSV files is and only is what you state (an internal intermediate format), then there is a mismatch between that and what a new user expects it to be. CSV files in many tools, including other 2-way radio tools, are an INTERCHANGE format. In particular, you can get CSV files with channel settings from other 2-way radio programming tools and sometimes as published with standards, repeater lists, or communications plans (e.g. the NIFOG CSVs). They often come in organized in different bands and zones.
A new user staring at a bunch of unprogrammed radios and a bunch of data from different sources is going to wonder how to get the data into the radio, preferably without having to hand program several hundred channels plus the non-standard customizations Bob and George want. They see that CHIRP uses CSV files, they know that have a pile of CSV files from/this or that source and they see a match made in heaven. How do I make a laydown that stuffs all of my NIFOG UHF simplex mutual-aid channels on top of the SKYWarn and ARES repeater channels form local group and maybe be able to listen to MURS or GMRS or whatever.
The logical thought process is that I can pull these files into something (often Excel or OO), do some adding, deleting, and moving of rows and spit out a bigger CSV with what I want to build my image. Preferably, you want some automation (which CHIRP does not provide) so that when one piece changes, you don't have to do the whole process over as well as maybe the chance to maintain the laydown and the table in the communications plan from one source. That is usually a forlorn hope, especially to somone not versed in UNIX Command-line tools for taking apart and recombining inputs from several sheets (John gets Zone C like Jane, but his transmitter is locked; John's Zone M is management and logistics using some group codes the other radios wont have, etc). But is a very reasonable expectation.
One of the best ways I have found to get close is to script transformations in Python so that I am at least using the same CSV library Chirp is using internally to minimize incompatibility. It would be extremely helpful however, if Chirp had a slightly more robust format *intended* for interchange and manipulation.
Hi Eric,
I am a fairly advanced user of Chirp myself at this point, but to me it is easy to understand how a new user gets stuck in this mess because I've done it myself as a new user often enough.
Sure, the average user thinks "A CSV file is a CSV file, so I should be able to import it just like I can import a .DOC file into OpenOffice."
However, you know this is not actually how it is. It's like saying "Microsoft Word supports XML files." It surely does, but only XML files formatted in *very* specific ways :)
The logical thought process is that I can pull these files into something (often Excel or OO), do some adding, deleting, and moving of rows and spit out a bigger CSV with what I want to build my image.
...snip...
It would be extremely helpful however, if Chirp had a slightly more robust format *intended* for interchange and manipulation.
If you would like to implement an Excel-like import dialog that allows you to customize everything about the import, including how to interpret columns that are titled and formatted in ways that CHIRP doesn't understand, please do. Patches are definitely welcome.
As you probably know, something like Excel has logic that tries to interpret lots of different formats for the few data types that it supports (number, date, currency, etc). It gets it right sometimes and wrong others. We have data types that are enumerations, and for which there could be lots of different methods of notation. Duplex could be "+/-/None/Split" like chirp uses, or "positive/negative/none" or "up/down/neither", or be missing entirely for something like a commercial band plan. PL codes could be listed in motorola syntax, by frequency, etc. Power levels could be in dBm, watts, "high/med/low" etc. The more of these you support, especially without known-named columns, the harder it is to make it "just happen" for most people. If you want to work on making this better, please (please) feel free to send patches :)
The biggest difference between Excel and CHIRP is, of course, the size of the budget and the funding of the team. CHIRP developers implement things that they want to implement, with no compensation. Almost nobody works on the UI of CHIRP because doing so is complex and time-consuming. Interpreting CSV files of more varied formats will involve a lot more UI-level work to allow the user to customize how CHIRP interprets the data. If this is what motivates you, please feel free to work on it.
Thanks!
--Dan
Okay -- here's the thing -- to be useful, computers and software should always make things easier for us humans, not the other way around. I've been doing this for a long time with various programming applications (such as RT Systems or FTBasicMMO and others) to program radios and I have ALWAYS been able to use Excel CSV files and NEVER had as much trouble with any software as I've had with Chirp--the CSV files just import and apparently the programming software ignores what it doesn't like.
Now I am experimenting with Chirp because I did not want to put out another $45 bucks to RT Systems, that's it -- I'm a cheapskate. Granted, Chirp is WAY more friendly and useful than the standard Wouxun programming software.
I already have numerous CSV files with all the repeater data that I need for various applications, some local, some in cities where I travel and I wanted to use this, not start from scratch entering data one line at a time into another programming tool. It's just easier to keep everything in one format -- essentially a database -- that I can carve up and use any way I want.
So, it's been a steep learning curve for me, and I did not mean to ruffle so many feathers, but I think I'm up to speed now and Chirp is now another tool in the toolbox...so to speak.
Regards, Chris W7CFN
On 2/27/2016 9:32 PM, Tom Hayward wrote:
On Sat, Feb 27, 2016 at 7:52 PM, Chris Nordstrom cfnordstrom@gmail.com wrote:
Right, well as I've said, when you create generic CSV file from Chirp and go look at it, it still contains all the empty fields and the extra commas at the end of the row. So....
The "extra" commas are a critical, required part of the Chirp CSV format.
I'm having a hard time understanding why you need Excel in the first place. It sounds like you are a relatively new user of Chirp, yet using third party software to edit Chirp files is arguably one of the most advanced things you can do with Chirp. As you've found, it requires understanding the specific details of the file formats and any errors that might be introduced by specific third party editors. A lot of effort has been put into the built-in Chirp editor. It supports copy, paste, inserting rows, removing rows, populating safe default values, automatic repeater offsets for various band plans around the world, bulk row editing (i.e., setting power level to high on 20 channels at once). What is missing from Chirp that drove you to an external editor? I'm not implying there are no valid uses of external editors--indeed, this is one of the benefits of using CSV encoding--but most of the time it just makes things more difficult.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
If the other info you want to use is on radios supported by CHIRP, just read those radios and save in CHIRP. Then read the radio you want to program, save the info just in case, then copy and paste from the first radio to the second. Very easy to do. Don't use Excel anymore than you absolutely need to. Do your editing within CHIRP.
Jardy Dawson WA7JRD Ham Radio
Message relayed through SETI (Search for Extra-Terrestrial Intelligence, although no terrestrial intelligence has been found yet, either)
On Feb 28, 2016, at 18:14, Chris Nordstrom cfnordstrom@gmail.com wrote:
Okay -- here's the thing -- to be useful, computers and software should always make things easier for us humans, not the other way around. I've been doing this for a long time with various programming applications (such as RT Systems or FTBasicMMO and others) to program radios and I have ALWAYS been able to use Excel CSV files and NEVER had as much trouble with any software as I've had with Chirp--the CSV files just import and apparently the programming software ignores what it doesn't like.
Now I am experimenting with Chirp because I did not want to put out another $45 bucks to RT Systems, that's it -- I'm a cheapskate. Granted, Chirp is WAY more friendly and useful than the standard Wouxun programming software.
I already have numerous CSV files with all the repeater data that I need for various applications, some local, some in cities where I travel and I wanted to use this, not start from scratch entering data one line at a time into another programming tool. It's just easier to keep everything in one format -- essentially a database -- that I can carve up and use any way I want.
So, it's been a steep learning curve for me, and I did not mean to ruffle so many feathers, but I think I'm up to speed now and Chirp is now another tool in the toolbox...so to speak.
Regards, Chris W7CFN
On 2/27/2016 9:32 PM, Tom Hayward wrote:
On Sat, Feb 27, 2016 at 7:52 PM, Chris Nordstrom cfnordstrom@gmail.com wrote: Right, well as I've said, when you create generic CSV file from Chirp and go look at it, it still contains all the empty fields and the extra commas at the end of the row. So....
The "extra" commas are a critical, required part of the Chirp CSV format.
I'm having a hard time understanding why you need Excel in the first place. It sounds like you are a relatively new user of Chirp, yet using third party software to edit Chirp files is arguably one of the most advanced things you can do with Chirp. As you've found, it requires understanding the specific details of the file formats and any errors that might be introduced by specific third party editors. A lot of effort has been put into the built-in Chirp editor. It supports copy, paste, inserting rows, removing rows, populating safe default values, automatic repeater offsets for various band plans around the world, bulk row editing (i.e., setting power level to high on 20 channels at once). What is missing from Chirp that drove you to an external editor? I'm not implying there are no valid uses of external editors--indeed, this is one of the benefits of using CSV encoding--but most of the time it just makes things more difficult.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Jardy at jardy72@yahoo.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
So ...
So, this "generic" file will presumably import into CHIRP without any dramas.
But it's not a "generic file" - it's an export of the setup that was in CHIRP at the time it was created, in a format that CHIRP can handle. CHIRP is just using the CSV protocol to hold its export.
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: Discussion of CHIRP Sent: Sunday, February 28, 2016 2:52 PM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Right, well as I've said, when you create generic CSV file from Chirp and go look at it, it still contains all the empty fields and the extra commas at the end of the row. So....
Regards, Chris W7CFN
On 2/27/2016 2:31 PM, Tom Hayward wrote:
On Feb 27, 2016 13:48, "Chris Nordstrom" cfnordstrom@gmail.com wrote: > I am not aware of these bugs in Excel that were mentioned. After creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in rToneFreq? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Yes, you need to provide appropriate defaults if you're not editing with Chirp (when you edit a CSV with Chirp, it will fill in all the defaults for you). Here's a quote from the documentation I linked earlier:
"Field values must be in the original format and one of the accepted values for the column type."
Excel is a generic spreadsheet program. If you want to use it to edit Chirp CSV files, you will need to handle the data validation yourself to ensure it stays in a format compatible with Chirp.
Tom KD7LXL
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11711 - Release Date: 02/27/16
I am not aware of these bugs in Excel that were mentioned. ...
There are (or were) bugs in Excel for CSV files, but these are not them. :-)
Excel will write a CSV file that contains the information it knows to put in. It will get this from two places:
1) Cells that came from a CSV file, 2) Cells that you've put values in.
So, the first 16 entries came in from the CSV file when it was read. These lines had included the 5 trailing commas, so Excel knew there were empty cells in those rows that needed to be written out.
The remainder of the entries came from extra rows that you added (I assume). There is no formal layout to a worksheet, so Excel has no way of guessing that you want all rows to have the same number of columns, or even that these rows are supposed to be the same width as what's in row 1 of the worksheet, which is simply another row of information.
I certainly couldn't put it any better than Tom, so: " It sounds like you are a relatively new user of Chirp, yet using third party software to edit Chirp files is arguably one of the most advanced things you can do with Chirp."
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Sunday, February 28, 2016 8:48 AM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Tom,
Okay, I just tested it again and it works as you say. When I was getting the import errors before--caused by not having data in key fields--it appeared that it stopped at the default number--25--which was only limiting what I could see, not what was imported. So it works fine.
I am not aware of these bugs in Excel that were mentioned. After creating my CSV file, I used a raw file viewer and did not see any extra characters or missing characters. I am using Excel now to edit the CSV files and they import fine. Is it true that there has to be a default value in rToneFreq? And what about the errors I get when there are columns with header text but no data in the column? Could this be caused by Excel?
Regards, Chris W7CFN
On 2/27/2016 11:57 AM, Tom Hayward wrote:
On Sat, Feb 27, 2016 at 11:33 AM, Chris Nordstrom cfnordstrom@gmail.com wrote: But I could not find where it says you should delete column header text for columns that contain no data and it also does not say that the Memory Range entry on the edit window must match the number of rows you are importing; the error you get doesn't lead you make this connection. Did I miss this too? Not sure what you mean about memory range view being related to the number of rows imported. I've imported outside the currently selected range many times and never had an issue. I just did it again now to verify. The memory range view is just a view; it doesn't affect imports.
Regarding the Excel bugs, there's only so much we can do about people using 3rd party software to edit Chirp files and getting unexpected results. As I said in my first email, I recommend editing Chirp files with Chirp. When you edit with Chirp, it maintains all the proper data in the proper columns. There is no need to delete headers or fill your own defaults--it's all automatic in Chirp. Feel free to write some documentation about the caveats you discovered when editing Chirp files in Excel.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11709 - Release Date: 02/27/16
If you enter a simplex freq with no tone required and do not fill in the other columns as shown that channel will not upload to the radio. Found that problem with a Baofeng UV5r as well.
Bottom line is if you fill in the columns as shown below for all channels with the appropriate data as required or the generic data as shown they will upload to the radio.
Location
Name
Frequency
Duplex
Offset
Tone
rToneFreq
cToneFreq
DtcsCode
DtcsPolarity
Mode
TStep
Skip
Comment
URCALL
RPT1CALL
RPT2CALL
1
SIM-52
146.52
0
88.5
88.5
23
NN
FM
5
From: chirp_users-bounces@intrepid.danplanet.com [mailto:chirp_users-bounces@intrepid.danplanet.com] On Behalf Of Chris Nordstrom Sent: Saturday, February 27, 2016 12:17 PM To: chirp_users@intrepid.danplanet.com Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Dave,
Yup, success! When I put 88.5 in the rToneFreq column (also used 67, I guess any valid tone freq) and deleted column headers from any column that contains no data and entered the correct number of rows in Memory Range---badda bing -- it works.
I also figured out that I cannot directly Upload a CSV file to the radio, it has to be an IMG file, which makes sense, but the method for getting an IMG file is not obvious and with no documentation (that I could find), eew! So what I did was Download from the radio, delete all except the first row, save it as an "Empty" IMG file. Next, I Open that IMG file then Import the CSV file containing the memory channel data I want, and magically, it becomes an IMG file that I can now Upload to the radio. Is this how it is supposed to work or is there another way to do this? Does it work basically the same regardless of the radio brand?
Attached is the CSV file that causes the Column number mismatch error. The only difference between this file and the one that successfully imports is that there IS column heading text in columns M, N, O, P and Q, but there is no data in those columns. When I removed the heading text from just those columns, the problem goes away.
Thanks for your encouragement and ideas,
Regards, Chris W7CFN
P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me.
On 2/26/2016 10:43 PM, Dave VK2FDJS wrote:
I assume the file you attached was the one that gave you the second (smaller) set of errors.
If you look at it in Excel, you'll see that rows 33, 35 and 40 are the only rows that have nothing in the rToneFreq and cToneFreq columns.
There are two other rows that have nothing in the Tone column (rows 2 and 3), but these still have values in the other two columns. Looks like you have to have a value of some sort in these two columns.
Could you please post the file that gave you the errors from row 17 on, so we can have a look at it too.
Regards, Dave VK2FDJS
----- Original Message -----
From: Chris Nordstrom mailto:cfnordstrom@gmail.com
To: chirp_users@intrepid.danplanet.com
Sent: Saturday, February 27, 2016 4:15 PM
Subject: Re: [chirp_users] Chirp won't read CSV files correctly
This is the error I get when importing or opening my CSV file:
I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time.
So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one:
The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out.
I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations.
Chris Nordstrom
_____
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
_____
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11703 - Release Date: 02/26/16
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
On Sat, Feb 27, 2016 at 1:17 PM, Chris Nordstrom cfnordstrom@gmail.com wrote:
Dave,
Yup, success! When I put 88.5 in the rToneFreq column (also used 67, I guess any valid tone freq) and deleted column headers from any column that contains no data and entered the correct number of rows in Memory Range---badda bing -- it works.
Back when I was using MS Office, I found out that a workaround for Excel's issue with multiple trailing commas (,) in a CSV was to make sure something was in the last column even if it was nothing more than a pair of double quotes ("").
Jim KC9HI
If you are editing CSV files, it is important that you use a text editor to at least inspect the results of the edit. In the event of problems using a CSV file modified in a third-party program (e.g. Excel), it becomes VITAL that you inspect the file in a text editor.
CSV files are text files, not Excel files. Excel can make use of them, but there is no reason to expect it to maintain a presupposed file format. In particular, there is no concept in a CSV file of a header row naming columns or a fixed number of "columns" per line.
In your case, here are the two rows at the position of the problem:
15,SARO-U,444.1,+,5,Tone,179.9,88.5,23,NN,FM,5,,,,, 16,SARO-U,444.125,+,5,Tone,179.9,88.5,23,NN,FM,5
As you can see, the line beginning with "15," which is the 16th line in the file, ends with a series of commas. These separate the last 5 values in the line, which all happen to be empty.
The line beginning with "16," - and subsequent lines - are lacking these 5 commas and accordingly contain five fewer values per line. Because CHIRP is expecting more values per line than you gave it, it doesn't know what to do, and hands the problem back to you with the "column number mismatch" error message. (Certainly it would be nice if the error message was a bit more informative - e.g. by including the number of values expected and the number of values actually in the line - perhaps the programmers can improve the error message.)
When I look at the file from yesterday (from which you had removed column headings), there are no trailing commas in any of the lines. I assume these were removed by Excel, but can't know for sure.
As a simple rule of thumb, only work on CSV files in the program that actually created them. If you do this, you are unlikely to have problems with them. As I understand it, the idea of CHIRP's CSV files is to allow you to copy entire sets of frequencies from one radio image to another, not to allow you to manipulate the frequency sets in third-party programs like Excel. (Like all rules, this is, of course, meant to be broken ... but caveat emptor.)
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Sunday, February 28, 2016 5:17 AM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Dave,
Yup, success! When I put 88.5 in the rToneFreq column (also used 67, I guess any valid tone freq) and deleted column headers from any column that contains no data and entered the correct number of rows in Memory Range---badda bing -- it works.
I also figured out that I cannot directly Upload a CSV file to the radio, it has to be an IMG file, which makes sense, but the method for getting an IMG file is not obvious and with no documentation (that I could find), eew! So what I did was Download from the radio, delete all except the first row, save it as an "Empty" IMG file. Next, I Open that IMG file then Import the CSV file containing the memory channel data I want, and magically, it becomes an IMG file that I can now Upload to the radio. Is this how it is supposed to work or is there another way to do this? Does it work basically the same regardless of the radio brand?
Attached is the CSV file that causes the Column number mismatch error. The only difference between this file and the one that successfully imports is that there IS column heading text in columns M, N, O, P and Q, but there is no data in those columns. When I removed the heading text from just those columns, the problem goes away.
Thanks for your encouragement and ideas,
Regards, Chris W7CFN
P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me.
On 2/26/2016 10:43 PM, Dave VK2FDJS wrote:
I assume the file you attached was the one that gave you the second (smaller) set of errors.
If you look at it in Excel, you'll see that rows 33, 35 and 40 are the only rows that have nothing in the rToneFreq and cToneFreq columns.
There are two other rows that have nothing in the Tone column (rows 2 and 3), but these still have values in the other two columns. Looks like you have to have a value of some sort in these two columns.
Could you please post the file that gave you the errors from row 17 on, so we can have a look at it too.
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Saturday, February 27, 2016 4:15 PM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
This is the error I get when importing or opening my CSV file:
I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time.
So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one:
The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out.
I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations.
Chris Nordstrom
-------------------------------------------------------------------------- _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
--------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11703 - Release Date: 02/26/16
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11706 - Release Date: 02/27/16
Dave,
I believe you about what Chirp is expecting, but when I run Chirp, choose /*File/New*/ to create a new empty CSV file, then I save that file, the resulting CSV file contains the extra commas at the end of the row. I use FileViewer to look at the raw text; this file has not been touched by Excel, yet contains the same extra commas.
I already discovered that if I delete the unused columns in Excel and re-import, the column number mismatch error is gone.
And I have to disagree-- the whole benefit of a program like this is to be able to manipulate (change) some data in order to customize my memory list, and I bet that's what most users are looking for.
Regards, Chris W7CFN
On 2/27/2016 2:30 PM, Dave VK2FDJS wrote:
If you are editing CSV files, it is important that you use a text editor to at least inspect the results of the edit. In the event of problems using a CSV file modified in a third-party program (e.g. Excel), it becomes VITAL that you inspect the file in a text editor. CSV files are text files, not Excel files. Excel can make use of them, but there is no reason to expect it to maintain a presupposed file format. In particular, there is no concept in a CSV file of a header row naming columns or a fixed number of "columns" per line. In your case, here are the two rows at the position of the problem: 15,SARO-U,444.1,+,5,Tone,179.9,88.5,23,NN,FM,5,,,,, 16,SARO-U,444.125,+,5,Tone,179.9,88.5,23,NN,FM,5 As you can see, the line beginning with "15," which is the 16th line in the file, ends with a series of commas. These separate the last 5 values in the line, which all happen to be empty. The line beginning with "16," - and subsequent lines - are lacking these 5 commas and accordingly contain five fewer values per line. Because CHIRP is expecting more values per line than you gave it, it doesn't know what to do, and hands the problem back to you with the "column number mismatch" error message. (Certainly it would be nice if the error message was a bit more informative - e.g. by including the number of values expected and the number of values actually in the line - perhaps the programmers can improve the error message.) When I look at the file from yesterday (from which you had removed column headings), there are no trailing commas in any of the lines. I assume these were removed by Excel, but can't know for sure. As a simple rule of thumb, only work on CSV files in the program that actually created them. If you do this, you are unlikely to have problems with them. As I understand it, the idea of CHIRP's CSV files is to allow you to copy entire sets of frequencies from one radio image to another, not to allow you to manipulate the frequency sets in third-party programs like Excel. (Like all rules, this is, of course, meant to be broken ... but caveat emptor.)
Regards, Dave VK2FDJS
----- Original Message ----- *From:* Chris Nordstrom <mailto:cfnordstrom@gmail.com> *To:* chirp_users@intrepid.danplanet.com <mailto:chirp_users@intrepid.danplanet.com> *Sent:* Sunday, February 28, 2016 5:17 AM *Subject:* Re: [chirp_users] Chirp won't read CSV files correctly Dave, Yup, success! When I put 88.5 in the */rToneFreq/* column (also used 67, I guess any valid tone freq) _/and/_ deleted column headers from any column that contains no data _/and/_ entered the correct number of rows in *Memory Range*---badda bing -- it works. I also figured out that I cannot directly /*Upload*/ a CSV file to the radio, it has to be an IMG file, which makes sense, but the method for getting an IMG file is not obvious and with no documentation (that I could find), eew! So what I did was */Download/* from the radio, delete all except the first row, save it as an "Empty" IMG file. Next, I */Open/* that IMG file then /*Import*/ the CSV file containing the memory channel data I want, and magically, it becomes an IMG file that I can now /*Upload*/ to the radio. Is this how it is supposed to work or is there another way to do this? Does it work basically the same regardless of the radio brand? Attached is the CSV file that causes the Column number mismatch error. The only difference between this file and the one that successfully imports is that there*IS* column heading text in columns M, N, O, P and Q, but there is no data in those columns. When I removed the heading text from just those columns, the problem goes away. Thanks for your encouragement and ideas, Regards, Chris W7CFN P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me. On 2/26/2016 10:43 PM, Dave VK2FDJS wrote:I assume the file you attached was the one that gave you the second (smaller) set of errors. If you look at it in Excel, you'll see that rows 33, 35 and 40 are the only rows that have nothing in the rToneFreq and cToneFreq columns. There are two other rows that have nothing in the Tone column (rows 2 and 3), but these still have values in the other two columns. Looks like you have to have a value of some sort in these two columns. Could you please post the file that gave you the errors from row 17 on, so we can have a look at it too. Regards, Dave VK2FDJS ----- Original Message ----- *From:* Chris Nordstrom <mailto:cfnordstrom@gmail.com> *To:* chirp_users@intrepid.danplanet.com *Sent:* Saturday, February 27, 2016 4:15 PM *Subject:* Re: [chirp_users] Chirp won't read CSV files correctly This is the error I get when importing or opening my CSV file: I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time. So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one: The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out. I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations. Chris Nordstrom ------------------------------------------------------------------------ _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com ------------------------------------------------------------------------ No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2015.0.6189 / Virus Database: 4537/11703 - Release Date: 02/26/16 _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris atcfnordstrom@gmail.com To unsubscribe, send an email tochirp_users-unsubscribe@intrepid.danplanet.com------------------------------------------------------------------------ _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com ------------------------------------------------------------------------ No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2015.0.6189 / Virus Database: 4537/11706 - Release Date: 02/27/16
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
I believe you about what Chirp is expecting, but when I run Chirp, choose File/New to create a new empty CSV file, then I save that file, the resulting CSV file contains the extra commas at the end of the row.
Yes, that would be correct. CHIRP includes the extra 5 commas, to match the full set of column labels in the first row.
I use FileViewer to look at the raw text; this file has not been touched by Excel, yet contains the same extra commas.
Remember that these are not "extra" commas. These are commas that CHIRP needs to have. The problem occurred when you imported a file that did NOT have these trailing commas in all lines, although the first 16 or so lines in the second file you posted did have the trailing commas.
When you deleted the unwanted column labels, for some reason Excel then deleted the trailing commas in the first 16 or so lines. Did you delete just the cells in the first row or did you use column delete in Excel?
And I have to disagree-- the whole benefit of a program like this is to be able to manipulate (change) some data in order to customize my memory list, and I bet that's what most users are looking for.
Well, I wasn't a developer, but my understanding of the purpose of CHIRP is to allow you to put the same setup onto multiple radios - potentially of different brands. The way it allows you to do this is by exporting the channel setup to a CSV file and then allowing you to import that CSV file into as many IMG files as you wish.
I suspect that the developers anticipated that any customisation of the memory list would actually be done in CHIRP, and then exported to CSV for import into the other IMG files. CSV files are not easy to play with (as you have found) and I doubt that the developers were anticipating that this would be the way that most users would edit their frequency data,
Yes, you can edit a CSV file directly, using Excel or simply a text editor. I do it myself for many CSV files. But I am an experienced IT person, with an interest in radio, so I know the types of things that can go wrong.
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Sunday, February 28, 2016 2:50 PM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Dave,
I believe you about what Chirp is expecting, but when I run Chirp, choose File/New to create a new empty CSV file, then I save that file, the resulting CSV file contains the extra commas at the end of the row. I use FileViewer to look at the raw text; this file has not been touched by Excel, yet contains the same extra commas.
I already discovered that if I delete the unused columns in Excel and re-import, the column number mismatch error is gone.
And I have to disagree-- the whole benefit of a program like this is to be able to manipulate (change) some data in order to customize my memory list, and I bet that's what most users are looking for.
Regards, Chris W7CFN
On 2/27/2016 2:30 PM, Dave VK2FDJS wrote:
If you are editing CSV files, it is important that you use a text editor to at least inspect the results of the edit. In the event of problems using a CSV file modified in a third-party program (e.g. Excel), it becomes VITAL that you inspect the file in a text editor.
CSV files are text files, not Excel files. Excel can make use of them, but there is no reason to expect it to maintain a presupposed file format. In particular, there is no concept in a CSV file of a header row naming columns or a fixed number of "columns" per line.
In your case, here are the two rows at the position of the problem:
15,SARO-U,444.1,+,5,Tone,179.9,88.5,23,NN,FM,5,,,,, 16,SARO-U,444.125,+,5,Tone,179.9,88.5,23,NN,FM,5
As you can see, the line beginning with "15," which is the 16th line in the file, ends with a series of commas. These separate the last 5 values in the line, which all happen to be empty.
The line beginning with "16," - and subsequent lines - are lacking these 5 commas and accordingly contain five fewer values per line. Because CHIRP is expecting more values per line than you gave it, it doesn't know what to do, and hands the problem back to you with the "column number mismatch" error message. (Certainly it would be nice if the error message was a bit more informative - e.g. by including the number of values expected and the number of values actually in the line - perhaps the programmers can improve the error message.)
When I look at the file from yesterday (from which you had removed column headings), there are no trailing commas in any of the lines. I assume these were removed by Excel, but can't know for sure.
As a simple rule of thumb, only work on CSV files in the program that actually created them. If you do this, you are unlikely to have problems with them. As I understand it, the idea of CHIRP's CSV files is to allow you to copy entire sets of frequencies from one radio image to another, not to allow you to manipulate the frequency sets in third-party programs like Excel. (Like all rules, this is, of course, meant to be broken ... but caveat emptor.)
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Sunday, February 28, 2016 5:17 AM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Dave,
Yup, success! When I put 88.5 in the rToneFreq column (also used 67, I guess any valid tone freq) and deleted column headers from any column that contains no data and entered the correct number of rows in Memory Range---badda bing -- it works.
I also figured out that I cannot directly Upload a CSV file to the radio, it has to be an IMG file, which makes sense, but the method for getting an IMG file is not obvious and with no documentation (that I could find), eew! So what I did was Download from the radio, delete all except the first row, save it as an "Empty" IMG file. Next, I Open that IMG file then Import the CSV file containing the memory channel data I want, and magically, it becomes an IMG file that I can now Upload to the radio. Is this how it is supposed to work or is there another way to do this? Does it work basically the same regardless of the radio brand?
Attached is the CSV file that causes the Column number mismatch error. The only difference between this file and the one that successfully imports is that there IS column heading text in columns M, N, O, P and Q, but there is no data in those columns. When I removed the heading text from just those columns, the problem goes away.
Thanks for your encouragement and ideas,
Regards, Chris W7CFN
P.S. Is there no one working on some basic documentation? Perhaps I can contribute something, it's been pretty painful for me.
On 2/26/2016 10:43 PM, Dave VK2FDJS wrote:
I assume the file you attached was the one that gave you the second (smaller) set of errors.
If you look at it in Excel, you'll see that rows 33, 35 and 40 are the only rows that have nothing in the rToneFreq and cToneFreq columns.
There are two other rows that have nothing in the Tone column (rows 2 and 3), but these still have values in the other two columns. Looks like you have to have a value of some sort in these two columns.
Could you please post the file that gave you the errors from row 17 on, so we can have a look at it too.
Regards, Dave VK2FDJS
----- Original Message ----- From: Chris Nordstrom To: chirp_users@intrepid.danplanet.com Sent: Saturday, February 27, 2016 4:15 PM Subject: Re: [chirp_users] Chirp won't read CSV files correctly
This is the error I get when importing or opening my CSV file:
I'm using a file that was generated by Chirp using the export function, then (In Excel) I copy/paste my channels into that CSV file. When opened in Chirp using either File/Open or File/Import, same errors every time.
So, a little experimentation and I discover that it doesn't like the empty columns to the right (where there is a column heading but nothing in the column). So I deleted the column headers (that Chirp created!) for the columns with no data and voila, it got past that error. Now I get this one:
The new error about line 33 "[rTone] could not convert string to float" I generally get what this error means but don't get what exactly is causing it. And why does it reference line 33 and only import up to memory channel 25? I notice at the top of the memory list window is an item called "Memories Memory Range" where the default entries are 1 to 25. Bingo! I changed the top end to 39 and it imported all the rows in my CSV file. Wow, I know, there is a disclaimer about how you guys are good at programming and not so much at documentation. Not sure I agree with that and the documentation part seems like a cop out.
I can't say never, but rarely have I encountered a program so unintuitive and persnickity. I know, I know, it's free, what do I want for nothing. I guess I need to lower my expectations.
Chris Nordstrom
---------------------------------------------------------------------- _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com ----------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11703 - Release Date: 02/26/16
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
-------------------------------------------------------------------------- _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
--------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11706 - Release Date: 02/27/16
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Chris at cfnordstrom@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Dave VK2FDJS at vk2fdjs@gmail.com To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
------------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.6189 / Virus Database: 4537/11711 - Release Date: 02/27/16
I think I understand why there is so much confusion -- my purpose was not to use a third party tool specifically for editing the data, it was because I already have numerous CSV files created by Excel or by export from another radio. I wanted to use one of these files to get my initial channel list into the new KG-UV8D without having to type in the data one row at a time.
That's where I started having trouble. As you say, those files did not contain the extra commas for the empty columns and that is why I got errors right off the bat. After deleting the extra columns (I just deleted the header text) and then Chirp was able to read and import the file without the mismatched column numbers error. Then I discovered that other fields must have a default number in it even if not used (rToneFreq for instance. I fixed that and now all is good.
Regards, Chris W7CFN
Ran into the same problems. Gave up on Chirp and just bought one of the better disk. Worked like a champ with no problems at all.KB7YIJ
On Friday, February 26, 2016 12:26 PM, Tom Hayward tom@tomh.us wrote:
On Fri, Feb 26, 2016 at 11:04 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
I have a KG-UV8D. I can use Chirp to read from the radio, and save the image and create a CSV file. But if I modify the CSV file and then try to open it from Chirp, I get various errors. I was able to successfully import 15 memories (plus the default 0 channel the Chirp creates -- don't know what this is), but when Chirp encounters the next line (16th entry) in my CSV file if barfs and says "Mismatched column numbers line 17".
Computers don't lie--I bet line 17 of your CSV file has mismatched line numbers, just as the error says. If you attach or post a link to your CSV file, I could tell for sure.
Excel has a known bug where it will remove empty cells from a CSV file. Maybe that's what did it. I generally don't recommend using anything other than Chirp to edit Chirp files.
More background reading here: http://chirp.danplanet.com/projects/chirp/wiki/CSV_HowTo
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Raymond at dnlrice@frontiernet.net To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
I have no problem with CHIRP on any of my radios including the UV8D.
Works like a champ!!!
From: chirp_users-bounces@intrepid.danplanet.com [mailto:chirp_users-bounces@intrepid.danplanet.com] On Behalf Of dnlrice@frontiernet.net Sent: Saturday, February 27, 2016 7:51 AM To: Discussion of CHIRP Subject: Re: [chirp_users] Chirp won't read CSV files correctly
Ran into the same problems. Gave up on Chirp and just bought one of the better disk. Worked like a champ with no problems at all.
KB7YIJ
On Friday, February 26, 2016 12:26 PM, Tom Hayward tom@tomh.us wrote:
On Fri, Feb 26, 2016 at 11:04 AM, Chris Nordstrom cfnordstrom@gmail.com wrote:
I have a KG-UV8D. I can use Chirp to read from the radio, and save the image and create a CSV file. But if I modify the CSV file and then try to open it from Chirp, I get various errors. I was able to successfully import 15 memories (plus the default 0 channel the Chirp creates -- don't know what this is), but when Chirp encounters the next line (16th entry) in my CSV file if barfs and says "Mismatched column numbers line 17".
Computers don't lie--I bet line 17 of your CSV file has mismatched line numbers, just as the error says. If you attach or post a link to your CSV file, I could tell for sure.
Excel has a known bug where it will remove empty cells from a CSV file. Maybe that's what did it. I generally don't recommend using anything other than Chirp to edit Chirp files.
More background reading here: http://chirp.danplanet.com/projects/chirp/wiki/CSV_HowTo
Tom KD7LXL
_______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to Raymond at dnlrice@frontiernet.net To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
The same thing happened to me with a KG-UV2D. I haven't been able to figure out why.
Tom
On 02/26/2016 13:04, Chris Nordstrom wrote:
I have a KG-UV8D. I can use Chirp to read from the radio, and save the image and create a CSV file. But if I modify the CSV file and then try to open it from Chirp, I get various errors.
Hi, Can you post the csv on the web or attach it to your message, we can have a better understanding of what's going on by having the file to look.
And yes Office 2003 has a bug with CSV, I have suffered from that in a previous work, I will suggest to use the free/libre suite LibreOffice.
73
El 26/02/16 a las 14:04, Chris Nordstrom escribió:
I have a KG-UV8D. I can use Chirp to read from the radio, and save the image and create a CSV file. But if I modify the CSV file and then try to open it from Chirp, I get various errors. I was able to successfully import 15 memories (plus the default 0 channel the Chirp creates -- don't know what this is), but when Chirp encounters the next line (16th entry) in my CSV file if barfs and says "Mismatched column numbers line 17".
I can delete the first 15 rows in my CSV file and create a new series of entries with the second half of my memory list, starting with Channel 16, and it works, until it gets to the 17th line, then issues the error "Mismatched column numbers line 17".
Now, I'm certain that import/export has to be a very basic function so I must be doing something wrong any help out there?
Chirp version (the only one I can find on the web site) : chirp-daily-20160224-installer.exe Windows 7 Microsoft Excel 2003
I also created an account on your the web site, but cannot login -- I get the message login info invalid-- do I have to wait for an email after creating an account?
Chris Nordstrom
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users This message was sent to CO7WT at co7wt@frcuba.co.cu To unsubscribe, send an email to chirp_users-unsubscribe@intrepid.danplanet.com
participants (12)
-
Chris Nordstrom -
Dan Smith -
Dave VK2FDJS -
dnlrice＠frontiernet.net -
Eric Vought -
Jardy Dawson -
Jim Unroe -
Ken Rugaber (KC9QV) -
Kevin Karzin -
Pavel Milanes (CO7WT) -
Tom Consodine ND5Y -
Tom Hayward