Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

352
Views
how to convert utf-8 to utf-16 with ndash?

both:

$result = iconv('UTF-8', 'UTF-16LE//IGNORE//TRANSLIT', $str);

and mb_convert_encoding() fail to convert a ndash (–) the long minus.

The result will go into a csv, so replacing it with an html entity is not an option. Any ideas?

code:

            $data = $eventHelper->getProgramForCsvExport($event);

            $response = new StreamedResponse();
            $response->setCharset('UTF-16LE');
            $filename = 'program-' . $event->getShortName() . $event->getShortYear() . '.csv';

            $utf16Data = [];
            foreach ($data as $row) {
                $utf16row = [];
               foreach ($row as $entry) {
                   $utf16row[] = iconv('UTF-8', 'UTF-16LE//IGNORE//TRANSLIT', $entry);
               }
                $utf16Data[] = $utf16row;

            }

            $response->setCallback(function () use ($utf16Data) {
                $output = fopen('php://output', 'w+');

                foreach ($utf16Data as $row) {
                    fputcsv($output, $row, ';');
                }

                fclose($output);
            });

            $response->headers->set('Content-Type', 'text/csv; charset=utf-16');
            $response->headers->set('Content-Disposition', 'attachment; filename="' . $filename . '"');

            return $response;

Edit: It's an export for the great office 365 which doesn't support UTF-8 anymore but is on UTF-16LE on default (as far as I read). German umlauts (äöüß) work fine (and didn't before the convert), but ndash (and maybe some other special chars) don't. Ndashes are either blank (on mac) or become brackets (or so) on windows.

over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

I think the problem is that your code isn't outputting a UTF-16LE BOM (byte order mark) at the beginning of the file, so the programs reading it don't know what encoding it's in and are (apparently) guessing poorly.

A UTF-16LE BOM is the byte sequence 0xFF 0xFE (in that order) right at the beginning of the file. Make that the first thing you write to your output. More about BOMs in this Unicode FAQ.

To test my theory, I wrote the byte sequence for a UTF-16LE file containing only the characters 0–0:

FF FE 30 00 13 20 30 00

The FF FE is the BOM, the 30 00 is the digit zero, the 13 20 is the N-dash, and the final 30 00 is the final digit zero. (The zeros are just there so I can easily find the dash, though in such a short file it wouldn't really be difficult.😀)

I was able to open that with Office 365 on Windows just fine.

Then I wrote a file without the BOM:

30 00 13 20 30 00

Office 365 did indeed misinterpret the N-dash and show it as a character that looks like a pair of brackets.

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!