Ошибка data truncation

I’m working with a fairly simple database, from a Java application. We’re trying to insert about 200k of text at a time, using the standard JDBC mysql adapter. We intermittently get a com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column error.

The column type is longtext, and database collation is UTF-8. The error shows up using both MyISAM and InnoDB table engines. Max packet size has been set ot 1 GB on both client and server sides, so that shouldn’t be causing a problem, either.

Kuro Neko's user avatar

asked Sep 16, 2008 at 14:44

Check that your UTF-8 data is all 3-byte Unicode. If you have 4-byte characters (legal in Unicode and Java, illegal in MySQL 5), it can throw this error when you try to insert them. This is an issue that should be fixed in MySQL 6.0.

answered Sep 16, 2008 at 15:11

Avi's user avatar

AviAvi

19.9k4 gold badges57 silver badges70 bronze badges

2

Well you can make it ignore the error by doing an INSERT IGNORE which will just truncate the data and insert it anyway. (from http://dev.mysql.com/doc/refman/5.0/en/insert.html )

If you use the IGNORE keyword, errors
that occur while executing the INSERT
statement are treated as warnings
instead. For example, without IGNORE,
a row that duplicates an existing
UNIQUE index or PRIMARY KEY value in
the table causes a duplicate-key error
and the statement is aborted. With
IGNORE, the row still is not inserted,
but no error is issued. Data
conversions that would trigger errors
abort the statement if IGNORE is not
specified. With IGNORE, invalid values
are adjusted to the closest values and
inserted; warnings are produced but
the statement does not abort.

answered Sep 16, 2008 at 14:52

Nicholas's user avatar

NicholasNicholas

5753 silver badges14 bronze badges

0

In mysql you can use MEDIUMTEXT or LONGTEXT field type for large text data

Satish Sharma's user avatar

answered Oct 16, 2012 at 12:02

grigson's user avatar

grigsongrigson

3,45829 silver badges20 bronze badges

It sounds to me like you are trying to put too many bytes into a column. I ran across a very similar error with MySQL last night due to a bug in my code. I meant to do

foo.status = 'inactive'

but had actually typed

foo.state = 'inactive'

Where foo.state is supposed to be a two character code for a US State ( varchar(2) ). I got the same error as you. You might go look for a similar situation in your code.

answered Sep 16, 2008 at 14:55

Aaron's user avatar

AaronAaron

2,4031 gold badge24 silver badges25 bronze badges

I just hit this problem and solved it by removing all the non-standard ascii characters in my text (following the UTF-8 advice above).

I had the problem on a Debian 4, Java 5 system; but the same code worked fine with Ubuntu 9.04, Java 6. Both run MySql 5.

answered Jul 10, 2009 at 16:06

Szemere's user avatar

SzemereSzemere

1881 silver badge5 bronze badges

Fixing a data truncated warning in MySQL

Recently, while working on a project, I ran into a warning telling me that my “data was truncated for” one of my columns when I was importing a CSV file into one of my SQL tables.

Data truncated warning in MySQL
Pictured: The error in question.

Concerned that I had done something wrong, I Googled for a solution. Unfortunately, I didn’t find any answers there, so I ended up having to find the source of this warning myself.

What does “data truncated” mean?

Truncated means “cut short”, and “data truncated” warnings or errors refer to a value’s data being cut off at the end during the importing process (e.g. a value of “2.9823” being imported as “2.98”). These errors are important warnings, because it notifies us that our data has not been imported accurately.

Data truncated warnings usually happen because the imported value:

  1. Does not match the data type of the column they are being imported into, e.g. inserting “120e” into an INT (i.e. integer) column will cause the data to be inserted as 120, truncating the e at the back.
  2. Exceeds the maximum length of the column, e.g. a string “abcdef” being imported into a VARCHAR(2) will be truncated into “ab”.

Which was why my problem was so perplexing.

The problem

I wrote an SQL query to load the CSV file on the left into my table on the right (pictured below):

The CSV and the imported table
The CSV and the resulting table.

This was the SQL query I used:

LOAD DATA INFILE '/path/to/test.csv'
INTO TABLE tmp
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(id,test);

All my values have been imported correctly to the table, so what exactly is being truncated?

After some research and asking around, I found the answer: the \r character at the end of “11” was being truncated.


Article continues after the advertisement:


The \r character

The \r character is part of the \r\n series of characters, used in Windows to denote a newline character. In all other operating systems, newlines are denoted using \n, but because the CSV file was generated in Windows, it uses \r\n for newlines instead.

showing newline character
You can reveal these hidden characters in Notepad++ by going to View > Show Symbol > Show End of Line. CR (carriage return) represents \r, while LF (line feed) represents \n.

Why \r was being read into the database

The \r character was being read into the database because of this line in my query:

LOAD DATA INFILE '/path/to/test.csv'
INTO TABLE tmp
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(id,test);

The first row in my data has the following characters:

1;11\r\n

Which was split into 1 and 11\r. As \r is not considered an integer, it cannot be entered into the test column, which is defined as an INTEGER. Hence, it got truncated.

To fix this warning, all I had to do was to change the problematic line in my SQL query:

LINES TERMINATED BY '\r\n'

And I would be fine even if I didn’t, because this is one of those rare cases where the data truncated warning is harmless!

Conclusion

In conclusion, this is an exploration of an interesting warning I had found while trying to import CSV files into a MySQL table. Remember to always be careful and check your queries before you start importing!

Leave a comment below if the article helped you!


Article continues after the advertisement:


I’m working with a fairly simple database, from a Java application. We’re trying to insert about 200k of text at a time, using the standard JDBC mysql adapter. We intermittently get a com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column error.

The column type is longtext, and database collation is UTF-8. The error shows up using both MyISAM and InnoDB table engines. Max packet size has been set ot 1 GB on both client and server sides, so that shouldn’t be causing a problem, either.

Kuro Neko's user avatar

asked Sep 16, 2008 at 14:44

Check that your UTF-8 data is all 3-byte Unicode. If you have 4-byte characters (legal in Unicode and Java, illegal in MySQL 5), it can throw this error when you try to insert them. This is an issue that should be fixed in MySQL 6.0.

answered Sep 16, 2008 at 15:11

Avi's user avatar

AviAvi

19.9k4 gold badges57 silver badges70 bronze badges

2

Well you can make it ignore the error by doing an INSERT IGNORE which will just truncate the data and insert it anyway. (from http://dev.mysql.com/doc/refman/5.0/en/insert.html )

If you use the IGNORE keyword, errors
that occur while executing the INSERT
statement are treated as warnings
instead. For example, without IGNORE,
a row that duplicates an existing
UNIQUE index or PRIMARY KEY value in
the table causes a duplicate-key error
and the statement is aborted. With
IGNORE, the row still is not inserted,
but no error is issued. Data
conversions that would trigger errors
abort the statement if IGNORE is not
specified. With IGNORE, invalid values
are adjusted to the closest values and
inserted; warnings are produced but
the statement does not abort.

answered Sep 16, 2008 at 14:52

Nicholas's user avatar

NicholasNicholas

5753 silver badges14 bronze badges

0

In mysql you can use MEDIUMTEXT or LONGTEXT field type for large text data

Satish Sharma's user avatar

answered Oct 16, 2012 at 12:02

grigson's user avatar

grigsongrigson

3,45829 silver badges20 bronze badges

It sounds to me like you are trying to put too many bytes into a column. I ran across a very similar error with MySQL last night due to a bug in my code. I meant to do

foo.status = 'inactive'

but had actually typed

foo.state = 'inactive'

Where foo.state is supposed to be a two character code for a US State ( varchar(2) ). I got the same error as you. You might go look for a similar situation in your code.

answered Sep 16, 2008 at 14:55

Aaron's user avatar

AaronAaron

2,4031 gold badge24 silver badges25 bronze badges

I just hit this problem and solved it by removing all the non-standard ascii characters in my text (following the UTF-8 advice above).

I had the problem on a Debian 4, Java 5 system; but the same code worked fine with Ubuntu 9.04, Java 6. Both run MySql 5.

answered Jul 10, 2009 at 16:06

Szemere's user avatar

SzemereSzemere

1881 silver badge5 bronze badges

Fix Data Is Truncated for a Column Error in MySQL

This article demonstrates the possible reasons and solutions for an error, saying that Data is truncated for a column in MySQL.

Fix Data is Truncated for a Column Error in MySQL

Here, we will explore the possible reasons and solutions to get rid of an error saying MySQL data is truncated for a column.

Data Is Too Large

There are some scenarios when we have to load the data from a file instead of inserting manually into the MySQL table. While loading, the Data truncated for a column 'columnName' at row # error may occur. Why?

This is because the given data seems too big for a data type of the column in the MySQL table. For instance, you have a table where a value attribute is of type text which can only accommodate 65K characters of the provided data.

You will get this error when you insert data for the value column that exceeds 65K.

To solve this problem, you need to change the type that can accept more data, such as MEDIUMTEXT or LONGTEXT, which can accommodate 16MB and 4GB. You can also find some optimization tips here.

We can also truncate the data using SET ANSI_WARNINGS OFF and insert the records again, considering the maximum length of the column’s data type.

Invalid Data Is Passed

The Error Code: 1265. Data truncated for column 'a' at row 1 also occurs in MySQL if we try to insert invalid data.

For example, we have a table where the price field is type float and accepts the NULL values. See the following:

#create a table named `prices`
CREATE TABLE prices (ID INT NOT NULL, price FLOAT NULL);
#insert the first record
INSERT prices VALUES (1, '');

Here, we insert an empty string in the price column, a float type that takes NULL values. Remember, NULL and '' are not the same thing.

Still, if we try to insert an empty string in a column that takes NULL values, it would generate the Error Code: 1265. Data truncated for column 'a' at row 1. The simplest solution to this problem is passing the correct value of the right data type as required by the respective column of the price table.

We can solve it by specifying the NULL values explicitly instead of an empty string ('') or do not even write the values for that particular column in the INSERT statement. See the following snippet considering our example.

INSERT prices VALUES (1, NULL); #insert null explicitly
INSERT prices (ID) VALUES (2); #don't specify values for `price` column

Incorrect Terminating Character

If we insert every line manually in the MySQL table, everything goes well. The error occurs when we try to load more than one line at once and don’t get the terminating character properly.

To solve this, we need to check the terminating character of the file and specify in the LOAD command as follows.

#if the terminating character is `tab`
FIELDS TERMINATED BY '\t'
#if the terminating character is a `comma`
FIELDS TERMINATED BY ','

Solution 1:[1]

Your problem is that at the moment your incoming_Cid column defined as CHAR(1) when it should be CHAR(34).

To fix this just issue this command to change your columns’ length from 1 to 34

ALTER TABLE calls CHANGE incoming_Cid incoming_Cid CHAR(34);

Here is SQLFiddle demo

Solution 2:[2]

I had the same problem because of an table column which was defined as ENUM(‘x’,’y’,’z’) and later on I was trying to save the value ‘a’ into this column, thus I got the mentioned error.

Solved by altering the table column definition and added value ‘a’ into the enum set.

Solution 3:[3]

By issuing this statement:

ALTER TABLES call MODIFY incoming_Cid CHAR;

… you omitted the length parameter. Your query was therefore equivalent to:

ALTER TABLE calls MODIFY incoming_Cid CHAR(1);

You must specify the field size for sizes larger than 1:

ALTER TABLE calls MODIFY incoming_Cid CHAR(34);

Solution 4:[4]

However I get an error which doesn’t make sense seeing as the column’s data type was properly modified?

| Level | Code | Msg | Warn | 12 | Data truncated for column 'incoming_Cid' at row 1

You can often get this message when you are doing something like the following:

REPLACE INTO table2 (SELECT * FROM table1);

Resulted in our case in the following error:

SQL Exception: Data truncated for column 'level' at row 1

The problem turned out to be column misalignment that resulted in a tinyint trying to be stored in a datetime field or vice versa.

Solution 5:[5]

In my case it was a table with an ENUM that accepts the days of the week as integers (0 to 6). When inserting the value 0 as an integer I got the error message «Data truncated for column …» so to fix it I had to cast the integer to a string. So instead of:

$item->day = 0;

I had to do;

$item->day = (string) 0;

It looks silly to cast the zero like that but in my case it was in a Laravel factory, and I had to write it like this:

$factory->define(App\Schedule::class, function (Faker $faker) {
    return [
        'day' => (string) $faker->numberBetween(0, 6),
        //
    ];
});

Solution 6:[6]

I had the same problem, with a database field with type «SET» which is an enum type.

I tried to add a value which was not in that list.

The value I tried to add had the decimal value 256, but the enum list only had 8 values.

1: 1   -> A
2: 2   -> B
3: 4   -> C
4: 8   -> D
5: 16  -> E
6: 32  -> F
7: 64  -> G
8: 128 -> H

So I just had to add the additional value to the field.

enter image description here

Reading this documentation entry helped me to understand the problem.

MySQL stores SET values numerically, with the low-order bit of the
stored value corresponding to the first set member. If you retrieve a
SET value in a numeric context, the value retrieved has bits set
corresponding to the set members that make up the column value. For
example, you can retrieve numeric values from a SET column like this:

mysql> SELECT set_col+0 FROM tbl_name; If a number is stored into a

If a number is stored into a SET column, the bits that are set in the
binary representation of the number determine the set members in the
column value. For a column specified as SET(‘a’,’b’,’c’,’d’), the
members have the following decimal and binary values.

SET Member  Decimal Value   Binary Value
    'a'                1          0001
    'b'                2          0010
    'c'                4          0100
    'd'                8          1000

If you assign a value of 9 to this column, that is 1001 in binary, so
the first and fourth SET value members ‘a’ and ‘d’ are selected and
the resulting value is ‘a,d’.

Solution 7:[7]

when i first tried to import csv into mysql , i got the same error , and then i figured out mysql table i created doesn’t have the character length of the importing csv field ,
so if it’s the first time importing csv

  1. its a good idea to give more character length .
  2. label all fields as varchar or text , don’t blend int or other values.

then you are good to go.

Solution 8:[8]

my issue was I used single quote instead of double quotes which add extra words in Boolean column

Solution 9:[9]

I Faced the same issue .In my case i was inserting a empty string for a numeric column.But by inserting a numeric value in string form for same column it got resolved e.g ‘12.56’ —> numeric column will work but » —> numeric column will give the above mentioned error.
Note: For numeric columns in MySql we can pass values in double quotes also .

Solution 10:[10]

Check whether you are using the ‘enum’ datatype.If you are using ‘enum’ datatype then the items inside the enum datatype should be exactly match with the data which you are entering.Ex.
You are taking the enum datatype like this:
enum(‘book’,stationery’,’others’)
then when you are inserting the data into the database you have to do like this:
INSERT INTO database_name.table_name (column1,…columnn) VALUES().
THE value should include same items which you are mentioned within the bracket of enum datatype.

Понравилась статья? Поделить с друзьями:
  • Ошибка debootstrap kali
  • Ошибка daewoo nexia p0141
  • Ошибка data folder not found как исправить
  • Ошибка data path
  • Ошибка debootstrap astra linux