Sign In Register

How can we help you today?

Start a new topic

Invalid string response character encoding (Spark.getHttp)

I have problem with CSV data fetched from Google Drive. It looks like Google sends data using UTF-8 encoding (sadly there are no headers with character encoding) but in GameSparks all national characters are broken - I suppose GameSparks for default use some kind of 8-bit encoding (for example ISO-8859-1).


Is there any option to set different default encoding when server doesn't provide info in headers or any method to transcode characters from one to another encoding?


Example code:

const url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vRxzVG6dcUsqC2g4VKGtkcy7uSKmrv3OgjMN5wYwJ1dm9bAlacU_yB2JDIW88bjY1eSv1nx5IcSjfNU/pub?output=csv';
const request = Spark.getHttp(url);
const response = request.get();
Spark.setScriptData('response', response.getResponseString());


Should be:

key,en,de,it,fr,es,pl
hell_world,Hello world,Hallo Welt,Ciao mondo,Bonjour le monde,Hola Mundo,Witaj świecie
good_morning,Good morning,Guten Morgen,Buongiorno,Bonjour,Buenos días,Dzień dobry


But is:

key,en,de,it,fr,es,pl\r\nhell_world,Hello world,Hallo Welt,Ciao mondo,Bonjour le monde,Hola Mundo,Witaj �wiecie\r\ngood_morning,Good morning,Guten Morgen,Buongiorno,Bonjour,Buenos días,Dzie� dobry

Indeed it's problem with importing UTF-8 text as 8-bit encoded data. Solution is a calculate correct code points by hand:


function fixUTF8(input) {
var output = '';
for (var i = 0, length_1 = input.length; i < length_1; ++i) {
var code = input.charCodeAt(i);
if (code >= 128) {
if (code < 224) {
var code2 = input.charCodeAt(++i);
code = ((code & 0x1f) << 6) | ((code2 & 0x3f));
}
else if (code < 240) {
var code2 = input.charCodeAt(++i);
var code3 = input.charCodeAt(++i);
code = ((code & 0xf) << 12) | ((code2 & 0x3f) << 6) | (code3 & 0x3f);
}
else {
var code2 = input.charCodeAt(++i);
var code3 = input.charCodeAt(++i);
var code4 = input.charCodeAt(++i);
code = ((code & 0x7) << 18) | ((code2 & 0x3f) << 12) | ((code3 & 0x3f) << 6) | (code4 & 0x3f);
}
}
output += String.fromCharCode(code);
}
return output;
}

Login to post a comment