[chirp_users] New probably rheas of CSV
Ahh the pitfalls of using an archaic mailing list instead of a forum, no ability to search old posts effectively.
I'm having an issue with CSV import. It seems that Chirp is using it's own interpretation.
1. Open Office and Excel both save CSV in formats that can not be imported which is not Chirps fault but I still have to deal with it. So, is there a list of settings and character encoding that actually work when saving from Open Office Calc?
2. Chirp seems to break the RFC 4180 ( http://tools.ietf.org/html/rfc4180 ) for CSV in that if you quote the header fields it breaks. Does that need to be reported as a bug?
This is what I'm doing that I cannot seem to get to work. I have a Calc document with the repeaters for the state of Indiana. I export a chirp file to csv and open that in calc as explained here: http://chirp.danplanet.com/issues/1251 Then I just copy and paste the fields into the chirp csv file and save. I get the no channels found error.
I use sed to strip out any excess hidden characters and then try again and still I get the no channels error.
Maybe I am going about this wrong? is there a way to post columns into chirp directly? If so that would be much easier but it doesn't seem to let me.
On Wed, Sep 24, 2014 at 10:41 AM, Bryan McWhirt bryan@kb9yen.com wrote:
Ahh the pitfalls of using an archaic mailing list instead of a forum, no ability to search old posts effectively.
Google does a pretty good job indexing the mailing lists. I find their search engine works better than any forum search feature.
I'm having an issue with CSV import. It seems that Chirp is using it's own interpretation.
- Open Office and Excel both save CSV in formats that can not be
imported which is not Chirps fault but I still have to deal with it. So, is there a list of settings and character encoding that actually work when saving from Open Office Calc?
- Chirp seems to break the RFC 4180 (
http://tools.ietf.org/html/rfc4180 ) for CSV in that if you quote the header fields it breaks. Does that need to be reported as a bug?
Yes, this is a regression, sorry. Chirp uses signatures found in files to identify the file type. When I added support for reading CSV files generated by various programming software, I changed Chirp-style CSV to be identified with this logic:
filename.endswith(".csv") and filedata.startswith("Location,")
This is done on raw files, before any CSV parsing, so it doesn't catch CSV variations such as quoted fields or semicolon field separators. It literally only matches "Location,". It was done this way to make file type detection very fast. Report a bug and I'll see what I can do about increasing the scope of things that are detected as CSV files.
http://chirp.danplanet.com/projects/chirp/repository/revisions/0a688afaca61/...
This is what I'm doing that I cannot seem to get to work. I have a Calc document with the repeaters for the state of Indiana. I export a chirp file to csv and open that in calc as explained here: http://chirp.danplanet.com/issues/1251 Then I just copy and paste the fields into the chirp csv file and save. I get the no channels found error.
A sample of the CSV you're having trouble with would be helpful. It's hard to see the problem without seeing the problem.
Tom KD7LXL
I'm attaching the csv file I have. Hopefully the list isn't set up to strip attachments.
KB9YEN Bryan McWhirt bryan@kb9yen.com
On 9/24/14, 2:13 PM, Tom Hayward wrote:
On Wed, Sep 24, 2014 at 10:41 AM, Bryan McWhirt bryan@kb9yen.com wrote:
Ahh the pitfalls of using an archaic mailing list instead of a forum, no ability to search old posts effectively.
Google does a pretty good job indexing the mailing lists. I find their search engine works better than any forum search feature.
I'm having an issue with CSV import. It seems that Chirp is using it's own interpretation.
- Open Office and Excel both save CSV in formats that can not be
imported which is not Chirps fault but I still have to deal with it. So, is there a list of settings and character encoding that actually work when saving from Open Office Calc?
- Chirp seems to break the RFC 4180 (
http://tools.ietf.org/html/rfc4180 ) for CSV in that if you quote the header fields it breaks. Does that need to be reported as a bug?
Yes, this is a regression, sorry. Chirp uses signatures found in files to identify the file type. When I added support for reading CSV files generated by various programming software, I changed Chirp-style CSV to be identified with this logic:
filename.endswith(".csv") and filedata.startswith("Location,")
This is done on raw files, before any CSV parsing, so it doesn't catch CSV variations such as quoted fields or semicolon field separators. It literally only matches "Location,". It was done this way to make file type detection very fast. Report a bug and I'll see what I can do about increasing the scope of things that are detected as CSV files.
http://chirp.danplanet.com/projects/chirp/repository/revisions/0a688afaca61/...
This is what I'm doing that I cannot seem to get to work. I have a Calc document with the repeaters for the state of Indiana. I export a chirp file to csv and open that in calc as explained here: http://chirp.danplanet.com/issues/1251 Then I just copy and paste the fields into the chirp csv file and save. I get the no channels found error.
A sample of the CSV you're having trouble with would be helpful. It's hard to see the problem without seeing the problem.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users
On Wed, Sep 24, 2014 at 11:34 AM, Bryan McWhirt bryan@kb9yen.com wrote:
I'm attaching the csv file I have. Hopefully the list isn't set up to strip attachments.
There are no encoding issues with this CSV. No quoted headers like you described in your report.
If you open this CSV in Chirp, you'll see in the debug log all the reported errors. For every line, it reports several errors: cTone is invalid rTone is invalid DTCS code is invalid DTCS polarity is invalid Mode is invalid Tuning step is invalid
The problem is you include all these columns in your CSV, but you don't provide sane values. If you're not going to enter your own values, you need to remove the column so that Chirp can insert its own defaults.
Tom KD7LXL
Ok, I was just using an exported csv as a template, I will remove those columns then.
KB9YEN Bryan McWhirt bryan@kb9yen.com
On 9/24/14, 3:00 PM, Tom Hayward wrote:
On Wed, Sep 24, 2014 at 11:34 AM, Bryan McWhirt bryan@kb9yen.com wrote:
I'm attaching the csv file I have. Hopefully the list isn't set up to strip attachments.
There are no encoding issues with this CSV. No quoted headers like you described in your report.
If you open this CSV in Chirp, you'll see in the debug log all the reported errors. For every line, it reports several errors: cTone is invalid rTone is invalid DTCS code is invalid DTCS polarity is invalid Mode is invalid Tuning step is invalid
The problem is you include all these columns in your CSV, but you don't provide sane values. If you're not going to enter your own values, you need to remove the column so that Chirp can insert its own defaults.
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users
On Wed, Sep 24, 2014 at 12:04 PM, Bryan McWhirt bryan@kb9yen.com wrote:
Ok, I was just using an exported csv as a template, I will remove those columns then.
The other option is to leave the defaults from that export in place instead of deleting them.
This is what Chirp saves when you save a blank CSV: Location,Name,Frequency,Duplex,Offset,Tone,rToneFreq,cToneFreq,DtcsCode,DtcsPolarity,Mode,TStep,Skip,Comment,URCALL,RPT1CALL,RPT2CALL 0,,146.010000,,0.600000,,88.5,88.5,023,NN,FM,5.00,,,,,,
Tom KD7LXL
Now that I have this working correctly I want to thank you all.
One question I have.
The list I have does not have offsets as they are all frequency default. Is there a way after I do the import to have Chirp set all the offisets automatically instead of doing it manually as they import to 'none' since there is not column for them.
KB9YEN Bryan McWhirt bryan@kb9yen.com
On 9/24/14, 3:06 PM, Tom Hayward wrote:
On Wed, Sep 24, 2014 at 12:04 PM, Bryan McWhirt bryan@kb9yen.com wrote:
Ok, I was just using an exported csv as a template, I will remove those columns then.
The other option is to leave the defaults from that export in place instead of deleting them.
This is what Chirp saves when you save a blank CSV: Location,Name,Frequency,Duplex,Offset,Tone,rToneFreq,cToneFreq,DtcsCode,DtcsPolarity,Mode,TStep,Skip,Comment,URCALL,RPT1CALL,RPT2CALL 0,,146.010000,,0.600000,,88.5,88.5,023,NN,FM,5.00,,,,,,
Tom KD7LXL _______________________________________________ chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users
On Wed, Sep 24, 2014 at 12:45 PM, Bryan McWhirt bryan@kb9yen.com wrote:
Now that I have this working correctly I want to thank you all.
One question I have.
The list I have does not have offsets as they are all frequency default. Is there a way after I do the import to have Chirp set all the offisets automatically instead of doing it manually as they import to 'none' since there is not column for them.
Currently, Chirp will only auto-set offsets for new channels.
Tom KD7LXL
What always works for me is to program a few sample channels into CHIRP and then export them to a CSV file. Then use the format of that file to create the new one.
Jim KC9HI
On Wed, Sep 24, 2014 at 1:41 PM, Bryan McWhirt bryan@kb9yen.com wrote:
Ahh the pitfalls of using an archaic mailing list instead of a forum, no ability to search old posts effectively.
I'm having an issue with CSV import. It seems that Chirp is using it's own interpretation.
- Open Office and Excel both save CSV in formats that can not be
imported which is not Chirps fault but I still have to deal with it. So, is there a list of settings and character encoding that actually work when saving from Open Office Calc?
- Chirp seems to break the RFC 4180 (
http://tools.ietf.org/html/rfc4180 ) for CSV in that if you quote the header fields it breaks. Does that need to be reported as a bug?
This is what I'm doing that I cannot seem to get to work. I have a Calc document with the repeaters for the state of Indiana. I export a chirp file to csv and open that in calc as explained here: http://chirp.danplanet.com/issues/1251 Then I just copy and paste the fields into the chirp csv file and save. I get the no channels found error.
I use sed to strip out any excess hidden characters and then try again and still I get the no channels error.
Maybe I am going about this wrong? is there a way to post columns into chirp directly? If so that would be much easier but it doesn't seem to let me.
-- KB9YEN Bryan McWhirt bryan@kb9yen.com
chirp_users mailing list chirp_users@intrepid.danplanet.com http://intrepid.danplanet.com/mailman/listinfo/chirp_users
participants (3)
-
Bryan McWhirt -
Jim Unroe -
Tom Hayward