Just another update

This commit is contained in:
Ingy döt Net 2015-02-20 00:35:01 -05:00
parent a25938f123
commit 00a190b0a6
6591 changed files with 94363 additions and 23227 deletions

View file

@ -1,5 +1,13 @@
In this task, the goal is to find the <em>character</em> and <em>byte</em> length of a string. This means encodings like [[UTF-8]] need to be handled properly, as there is not necessarily a one-to-one relationship between bytes and characters. By ''character'', we mean an individual Unicode ''code point'', not a user-visible ''grapheme'' containing combining characters. For example, the character length of "møøse" is 5 but the byte length is 7 in UTF-8 and 10 in UTF-16.
{{omit from|GUISS|Can use Microsoft Word document properties, but this might not be accurate}}
{{omit from|Openscad}}
In this task, the goal is to find the <em>character</em> and <em>byte</em> length of a string.
This means encodings like [[UTF-8]] need to be handled properly, as there is not necessarily a one-to-one relationship between bytes and characters. <br>
By ''character'', we mean an individual Unicode ''code point'', not a user-visible ''grapheme'' containing combining characters.
For example, the character length of "møøse" is 5 but the byte length is 7 in UTF-8 and 10 in UTF-16.
Non-BMP code points (those between 0x10000 and 0x10FFFF) must also be handled correctly: answers should produce actual character counts in code points, not in code unit counts. Therefore a string like "𝔘𝔫𝔦𝔠𝔬𝔡𝔢" (consisting of the 7 Unicode characters U+1D518 U+1D52B U+1D526 U+1D520 U+1D52C U+1D521 U+1D522) is 7 characters long, '''not''' 14 UTF-16 code units; and it is 28 bytes long whether encoded in UTF-8 or in UTF-16.
Non-BMP code points (those between 0x10000 and 0x10FFFF) must also be handled correctly: answers should produce actual character counts in code points, not in code unit counts.
Therefore a string like "𝔘𝔫𝔦𝔠𝔬𝔡𝔢" (consisting of the 7 Unicode characters U+1D518 U+1D52B U+1D526 U+1D520 U+1D52C U+1D521 U+1D522) is 7 characters long, '''not''' 14 UTF-16 code units; and it is 28 bytes long whether encoded in UTF-8 or in UTF-16.
Please mark your examples with <nowiki>===Character Length=== or ===Byte Length===</nowiki>. If your language is capable of providing the string length in graphemes, mark those examples with <nowiki>===Grapheme Length===</nowiki>. For example, the string "J̲o̲s̲é̲" ("J\x{332}o\x{332}s\x{332}e\x{301}\x{332}") has 4 user-visible graphemes, 9 characters (code points), and 14 bytes when encoded in UTF-8.
Please mark your examples with <nowiki>===Character Length=== or ===Byte Length===</nowiki>.
If your language is capable of providing the string length in graphemes, mark those examples with <nowiki>===Grapheme Length===</nowiki>. <br>
For example, the string "J̲o̲s̲é̲" ("J\x{332}o\x{332}s\x{332}e\x{301}\x{332}") has 4 user-visible graphemes, 9 characters (code points), and 14 bytes when encoded in UTF-8.

View file

@ -1,2 +1,4 @@
---
category:
- String manipulation
note: Basic language learning

View file

@ -1 +1,6 @@
#len("Hello World")#
<cfoutput>
<cfset str = "Hello World">
<cfset j = createObject("java","java.lang.String").init(str)>
<cfset t = j.getBytes()>
<p>#arrayLen(t)#</p>
</cfoutput>

View file

@ -0,0 +1,3 @@
name = "J\x{332}o\x{332}s\x{332}e\x{301}\x{332}"
byte_size(name)
# => 14

View file

@ -0,0 +1,3 @@
name = "J\x{332}o\x{332}s\x{332}e\x{301}\x{332}"
Enum.count(String.codepoints(name))
# => 9

View file

@ -0,0 +1,3 @@
name = "J\x{332}o\x{332}s\x{332}e\x{301}\x{332}"
String.length(name)
# => 4

View file

@ -1,4 +1,4 @@
String s = "Hello, world!";
int byteCountUTF16 = s.getBytes("UTF-16").length; // Incorrect it yield 16 that is with the BOM
int byteCountUTF16 = s.getBytes("UTF-16LE").length; // Correct it yield 14
int byteCountUTF8 = s.getBytes("UTF-8").length;
int byteCountUTF16 = s.getBytes("UTF-16").length; // Incorrect: it yields 28 (that is with the BOM)
int byteCountUTF16LE = s.getBytes("UTF-16LE").length; // Correct: it yields 26
int byteCountUTF8 = s.getBytes("UTF-8").length; // yields 13

View file

@ -0,0 +1 @@
length("Hello world");

View file

@ -0,0 +1 @@
nops(convert("Hello world",bytes));

View file

@ -0,0 +1,20 @@
:- module string_byte_length.
:- interface.
:- import_module io.
:- pred main(io::di, io::uo) is det.
:- implementation.
:- import_module list, string.
main(!IO) :-
Words = ["møøse", "𝔘𝔫𝔦𝔠𝔬𝔡𝔢", "J\x332\o\x332\s\x332\e\x301\\x332\"],
io.write_list(Words, "", write_length, !IO).
:- pred write_length(string::in, io::di, io::uo) is det.
write_length(String, !IO):-
NumBytes = count_utf8_code_units(String),
io.format("%s: %d bytes\n", [s(String), i(NumBytes)], !IO).

View file

@ -0,0 +1,20 @@
:- module string_character_length.
:- interface.
:- import_module io.
:- pred main(io::di, io::uo) is det.
:- implementation.
:- import_module list, string.
main(!IO) :-
Words = ["møøse", "𝔘𝔫𝔦𝔠𝔬𝔡𝔢", "J\x332\o\x332\s\x332\e\x301\\x332\"],
io.write_list(Words, "", write_length, !IO).
:- pred write_length(string::in, io::di, io::uo) is det.
write_length(String, !IO) :-
NumChars = count_codepoints(String),
io.format("%s: %d characters\n", [s(String), i(NumChars)], !IO).