Hi!

First of all I must say that the Importer module is the coolest plugin for EE! (really a lifesaver/timesaver sometimes)

But Sadly if I import an XLS file, my foreign (hungarian) characters are messed up. (e.g.: á, Á, ó, Ó, ő, Ő, ö, Ö, ú, Ú, í, Í, ü, Ü, ű, Ű)

I just see question marks where my foreign character should be. My site uses UTF-8 character encoding everywhere. (In the control panel, webserver (set in .htaccess), in the templates,  SQL server, etc.)

My question is: Is this a Bug in the Beta or I just missing something?

I really need this module to work right! Please Help!

p.s.: No problem with the XML import though, but I need XLS support anyway.

Thanks for yor reply in advance,
Lehel Seres

LeLehel,

Have you experimented with turning the “Convert High ASCII Characters to Entities” setting on? That’s how I normally deal with this very difficult challenge.

mk

Hi!

Thank you for your reply!

Yes, I tried turning “Convert High ASCII Characters to Entities” setting to on.

It’s working on right, BUT! When I switch back “Convert High ASCII Characters to Entities” setting to OFF, I see the ACSII codes in my weblog instead of the right foreign character. ( “&-#-225;” = “á” )

Why turn “Convert High ASCII Characters to Entities” back from on to off you ask? Because I have to use EE that way. You see when I query a weblog from the database I want to sort in alphabetical order and the “&-#-225;” code won’t sort right. The character “á” should be on the top of the list, but it isn’t because the database server stores the char “á” as “&-#-225;”. So sorting is useless this way.

Is it possible to fix importer to handle the foreign characters right? Or is there any other workaround?

Thanks,
LeLehel

Hi again!

And one more thing!

There is another problem with turning the “Convert High ASCII Characters to Entities” setting to ON.

When I import an XLS file with names into a weblog and if the last character of the name is a foreign character, importer will cut of that last character.

For example the name “Máté” will look like “Mát” without the “é” on the end. All this with “Convert High ASCII Characters to Entities” setting turned to ON. When I switch this option to “NO” I get this: “M�t�” So back to my original problem.

Albeit XML import works perfectly with foreign characters, sadly my client doesn’t want to use XML. :(

HELP!

Sorry for my bad english and Thanks,
LeLehel

LeLehel,

I’m afraid you’ve hit me squarely in my achilles heel. I’m awful about writing software for foreign characters. At the moment, until I can get more time, the best I can offer is the ASCII conversion or the XML import option. Microsoft formats tend to be pretty difficult to work with as well. I wish I could help more.

mk

Hi Mitchell!

My Achilles heel is also in pain! I play badminton 3 times a week, and I’m getting older each week. Oh man it hurts… 

Anyway, I’m sorry to hear that there is no solution for my problem. :(

I paid close to 80$ for importer and now mostly it’s useless for me. Not a cheap module by the way!

As I was reading through the solspace forums, I saw that there are other modules suffering from foreign character problems.

In this thread ” http://www.solspace.com/forums/viewthread/194/ ” the TAG module and the French language are having problem with each other. BUT! At the end of the thread I read:

“We’ve added tentative support for foreign characters now with the new version of Tag 2.5”

So you guys can solve foreign character problems! Albeit Tag is only a 40$ module. I think a 80$ module like importer also deserves foreign character support (for the rest of us)

What do you think? When will it happen?

Looking forward…

LeLehel

LeLehel,

Well you should definitely not have to pay for software that won’t work for you. But let’s give this a try. I was recently pointed to some helpful character encoding code that might get me out of my mental block. Is your site in a place where I could test and work directly in your environment? That’s the best way to resolve the issues. If not, could you simply send over the Excel file you’re using? In either case, send me a Private Message through the forums and I’ll have a further look.

mk

LeLehel,

Well we have a new version of Importer for you to try. Pie Man will launch in just a bit. It should help with character encoding on Excel imports. Here’s the important thing though. After making my changes to Importer, I had no luck for foreign characters and UTF-8. I had to switch the site to ISO-8859-1. Once I did, it was all love, peace and chicken grease.

mk

LeLehel,

A new version of Importer is now available for download

http://www.solspace.com/forums/viewthread/2174/

Hi Mitchell! Hi Pie Man!

Sorry for the late reply, I was out of town.

So I’m very happy to hear the good news. I’ll give it a try once I get back to the office.

> If not, could you simply send over the Excel file you’re using?

Sure I can! I just sent you a personal message with the link.

> Is your site in a place where I could test and work directly in your environment?

My client’s host is upgrading server in a few days, so I won’t nag that one, but I also tested Importer 1.0.8Beta on EngineHosting based personal site. Same problem there.

I’ll get back soon!

LeLehel

Thanks LeLehel,

I can confirm as well, that I can get this to work properly when using ISO-8859-1. You’ll need to keep your site at those settings though

Hi Guys!

I’m Back!

I tested the new build. Yes in ISO-8859-1 everything is working fine! And also ISO-8859-2 seems to be ok.

But I’m still sad because my database is in UTF-8 so sorting a query is still false. Second all of my pages are in UTF-8 so if I do a query on any page I get the messed up character. Maybe somehow I can display two different character encoding on one page!? With iframe perhaps, but it’s not a clean solution.

So I really appreciate the work you have done so far with foreign characters!!! Thank you again! But maybe you can push your self towards to your limits and find a final solution for importer for full UTF-8 support.

Yes, I can feel that you start to have enough of me.

But think about it!
UTF-8 is becoming a “de facto” standard in character encoding. Even US sites are starting to use UTF-8 instead of ISO-8859-1 or USASCII.

I feel it worth the effort for you guys to enhance this product for a wider market.

Now I’ll stop hurting you guys! I promise!

I think Importer is a great product with only a small disability.

Lelehel

Lelehel

We’re getting a better developer on it for you… haha

Lelehel,

Well our better developer didn’t work any miracles. The module is dependent on an Excel parsing library. That library does its best to handle a very difficult job—handling Microsoft formats. Right around here is where the library breaks down.

The work-around for this is for us to include a way for you to specify, per import, which character encoding to use. We won’t be able to enable that feature for a while. I’m sorry.

As always, we’re glad to refund your license fee if the software does not fit your needs.

mk

Hi!

I had a discussion with my client, and we agreed that CSV import could be just as good as an XLS.

What do you think? Is it hard for you to implement CSV support?

Yes I know, there is the free CSVgrab plugin for EE, but that is very inconvenient for them. I was using CSVgrab for my home projects and always get the job done but it was far-far from perfect. And in some server configurations CSVgrab just didn’t work.

Using CSV there is no character encoding problems koz it’s just a txt file encoded in whatever you like.

So it would be supercool if you guys could make importer to handle CSV!!! Is it possible? Pleeeease!

I can imagine that with the ease of use of importer CSV file import could be just as fun.

LeLehel

Lelehel,

CSV support is definitely on our list for Importer, but it won’t be ready right away. I’m confident that due to the complexity, it will not be ready in time to meet your client’s needs. I’m sorry about it.

mk