How to get values from MySQL(5.6) column if that contains json document as string

别来无恙 提交于 2019-11-27 19:23:44

To be able to do what you want to, you need MySQL 5.7.8+. Since 5.7.8 you can use JSON_EXTRACT function to extract a value from a JSON string:

SELECT JSON_EXTRACT('{"id": 14, "name": "Aztalan"}', '$.name');

+---------------------------------------------------------+
| JSON_EXTRACT('{"id": 14, "name": "Aztalan"}', '$.name') |
+---------------------------------------------------------+
| "Aztalan"                                               |
+---------------------------------------------------------+

Taken from here.

In MySQL 5.6 you just can't get the value you want as MySQL doesn't know anything about what a JSON object is. So your options are:

In MySQL 5.6, by default JSON_EXTRACT is not available by default.
If you still need to access json data in MySQL 5.6, you need to write custom function.

DELIMITER $$

DROP FUNCTION IF EXISTS `json_extract_c`$$

CREATE DEFINER=`root`@`%` FUNCTION `json_extract_c`(
  details TEXT,
  required_field VARCHAR (255)
) RETURNS TEXT CHARSET latin1
BEGIN
  RETURN TRIM(
    BOTH '"' FROM SUBSTRING_INDEX(
      SUBSTRING_INDEX(
        SUBSTRING_INDEX(
          details,
          CONCAT(
            '"',
            SUBSTRING_INDEX(required_field,'$.', - 1),
            '"'
          ),
          - 1
        ),
        '",',
        1
      ),
      ':',
      - 1
    )
  ) ;
END$$

DELIMITER ;

This will help. I have created it and tested.

Both previous answers didn't work for me when the element wasn't mentioned in JSON text. There is my improved function:

DELIMITER $$

DROP FUNCTION IF EXISTS `json_extract_c`$$

CREATE FUNCTION `json_extract_c`(
details TEXT,
required_field VARCHAR (255)
) RETURNS TEXT CHARSET latin1
BEGIN
  DECLARE search_term TEXT;
  SET details = SUBSTRING_INDEX(details, "{", -1);
  SET details = SUBSTRING_INDEX(details, "}", 1);
  SET search_term = CONCAT('"', SUBSTRING_INDEX(required_field,'$.', - 1), '"');
  IF INSTR(details, search_term) > 0 THEN
    RETURN TRIM(
      BOTH '"' FROM SUBSTRING_INDEX(
        SUBSTRING_INDEX(
          SUBSTRING_INDEX(
            details,
            search_term,
            - 1
          ),
          ',"',
          1
        ),
        ':',
        -1
      )
    );
  ELSE
    RETURN NULL;
  END IF;
END$$

DELIMITER ;

Here are my 3 SQL stored functions that I use for JSON extracting. They handle nested objects, but only care about the key name. The keys must be strings, the values can be strings, numeric or boolean values. Arrays are not treated well, only the first value is picked. They return NULL if no value found.

The first one, json_extract_1 picks only the first value if there are multiple keys with the same name. If you leave the LIMIT 1 clause, it throws 'Subquery returns more than 1 row' if more keys found (safe mode).

The second, json_extract_m collects all values with the same key in a comma-separated list.

The third one, json_extract_c is the slowest one, but it treats values with commas also correctly. Use it if it's absolutely necessary, e.g textual descriptions.

For all three the limitation is 999 keys. You can speed up if you prepare a table for the numbers subselect.

DELIMITER $$

/*
 * Single-value JSON extract - picks the first value
 */
DROP FUNCTION IF EXISTS `json_extract_1`$$
CREATE FUNCTION `json_extract_1`(json_txt TEXT, search_key VARCHAR (255)) 
    RETURNS TEXT
BEGIN
    RETURN (SELECT TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(txt,':',-1), '"', 2), '"', -1)) AS val
    FROM (
        SELECT TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX( SUBSTRING_INDEX(json_txt , ',', n), ',',  -1 ), '}', 1), '{', -1)) AS txt
        FROM (SELECT t1.v + t2.v*10 + t3.v*100 AS n
            FROM (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t1,
            (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t2,
            (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t3) numbers
        WHERE CHAR_LENGTH(json_txt ) - CHAR_LENGTH(REPLACE(json_txt , ',', '')) >= n - 1
        AND n>0 ) sp
    WHERE TRIM(SUBSTRING_INDEX(txt,':',1)) = CONCAT('"',search_key,'"')
    LIMIT 1   -- comment out for safe mode
    );
END$$

/*
 * Multi-value JSON extract - collects all values, group_concats them with comma
 */
DROP FUNCTION IF EXISTS `json_extract_m`$$
CREATE FUNCTION `json_extract_m`(json_txt TEXT, search_key VARCHAR (255)) 
    RETURNS TEXT
BEGIN
    RETURN (SELECT GROUP_CONCAT(TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(txt,':',-1), '"', 2), '"', -1))) AS val
    FROM (
        SELECT TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX( SUBSTRING_INDEX(json_txt , ',', n), ',',  -1 ), '}', 1), '{', -1)) AS txt
        FROM (SELECT t1.v + t2.v*10 + t3.v*100 AS n
            FROM (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t1,
            (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t2,
            (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t3) numbers
        WHERE CHAR_LENGTH(json_txt ) - CHAR_LENGTH(REPLACE(json_txt , ',', '')) >= n - 1
        AND n>0 ) sp
    WHERE TRIM(SUBSTRING_INDEX(txt,':',1)) = CONCAT('"',search_key,'"'));
END$$

/*
 * Comma-safe JSON extract - treats values with commas correctly, but slow like hell
 */
DROP FUNCTION IF EXISTS `json_extract_c`$$
CREATE FUNCTION `json_extract_c`(json_txt TEXT, search_key VARCHAR (255)) 
    RETURNS TEXT
BEGIN
    DROP TEMPORARY TABLE IF EXISTS json_parts;
    DROP TEMPORARY TABLE IF EXISTS json_parts2;
    DROP TEMPORARY TABLE IF EXISTS json_indexes;

    CREATE TEMPORARY TABLE json_parts AS
    SELECT n, IF(INSTR(txt,':')>0 AND (INSTR(txt,',')+INSTR(txt,'{')>0),1,0) AS this_val, IF(INSTR(txt,':')>0 AND (INSTR(txt,',')+INSTR(txt,'{')=0),1,0) AS next_val, IF(INSTR(txt,',')+INSTR(txt,'{')>0,1,0) AS next_key, txt
    FROM (SELECT n, IF(n%2,txt,REPLACE(txt,',','|')) AS txt 
    FROM (SELECT n, TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(json_txt , '"', n), '"',  -1 )) AS txt
        FROM (SELECT t1.v + t2.v*10 + t3.v*100 AS n
        FROM (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t1,
        (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t2,
        (SELECT 0 AS v UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) t3) numbers
        WHERE CHAR_LENGTH(json_txt ) - CHAR_LENGTH(REPLACE(json_txt , '"', '')) >= n - 1
        AND n>0) v
        ) v2;
    CREATE TEMPORARY TABLE json_parts2 AS SELECT * FROM json_parts;

    CREATE TEMPORARY TABLE json_indexes AS
    SELECT p1.n, p1.n+1 AS key_idx, MIN(GREATEST(IF(p2.this_val,p2.n,0), IF(p2.next_val,p2.n+1,0))) AS val_idx, p2.this_val AS trim_val
    FROM json_parts p1
    JOIN json_parts2 p2 ON (p1.n < p2.n AND (p2.this_val OR p2.next_val)) 
    WHERE p1.next_key
    GROUP BY p1.n;  

    RETURN (SELECT json_values.v 
        FROM (SELECT p1.txt AS k, REPLACE(IF(i.trim_val, regexp_replace(regexp_replace(p2.txt,'^[: {]+',''),'[, }]+$',''), p2.txt), '|', ',') AS v
            FROM json_indexes i
            JOIN json_parts p1 ON (i.key_idx = p1.n)
            JOIN json_parts2 p2 ON (i.val_idx = p2.n)) json_values
        WHERE json_values.k = search_key);
END$$

DELIMITER ;

Yep, and if you have the chance, try to upgrade to MySQL 5.7, the built-in functions work much more efficiently.

user3631341

Rahul's answer did not work quite well for me, so I edited it and this worked for me:

DELIMITER $$

DROP FUNCTION IF EXISTS `json_extract_c`$$

CREATE FUNCTION `json_extract_c`(
details TEXT,
required_field VARCHAR (255)
) RETURNS TEXT CHARSET latin1
BEGIN
SET details = TRIM(LEADING '{' FROM TRIM(details));
SET details = TRIM(TRAILING '}' FROM TRIM(details));
RETURN TRIM(
    BOTH '"' FROM SUBSTRING_INDEX(
        SUBSTRING_INDEX(
            SUBSTRING_INDEX(
                details,
                CONCAT(
                    '"',
                    SUBSTRING_INDEX(required_field,'$.', - 1),
                    '":'
                ),
                - 1
            ),
            ',"',
            1
        ),
        ':',
        -1
    )
) ;
END$$

DELIMITER ;

Below answer works for me. it remove double quotes from the value.

DELIMITER $$

DROP FUNCTION IF EXISTS `json_extract_values`$$

CREATE DEFINER=`root`@`localhost` FUNCTION `json_extract_values`(
  details TEXT,
  required_field VARCHAR (255)
) RETURNS TEXT CHARSET latin1
BEGIN
SET details = SUBSTRING_INDEX(details, "{", -1);
SET details = SUBSTRING_INDEX(details, "}", 1);
RETURN 
    SUBSTRING_INDEX(
        TRIM(
            TRAILING '"' FROM 
                SUBSTRING_INDEX(
                    details,
                    CONCAT( 
                           '"', 
                           SUBSTRING_INDEX(required_field,'$.', - 1),
                           '":'
                          ),
                -1 )
            ),
        '"',
     -1);    
END$$

DELIMITER ;

Function above not work properly if you have nested JSON in table field.

Because i needed a JSON_EXTRACT on mysql 5.6 i wrote it by myself a copy of original function that can extract values like the native function in mysql 5.7

Usage:

SELECT JSON_EXTRACT_NESTED(table_field,"json_level1.json_level2.json_level3") FROM table;

If you have one level JSON then you use:

SELECT JSON_EXTRACT_NESTED(table_field,"json_level1") FROM table;

In database you have to add two functions:

Main function:

CREATE FUNCTION `json_extract_nested`(
_field TEXT,
_variable TEXT
) RETURNS TEXT CHARSET latin1
BEGIN
                DECLARE X INT DEFAULT 0;
                DECLARE fieldval1 TEXT;
                DECLARE arrayName,arrayValue TEXT;

                SET arrayName = SUBSTRING_INDEX(_variable, '.', 1);

                IF(LOCATE('%',arrayName)> 0) THEN 
                    SET _field = SUBSTRING_INDEX(_field, "{", -1);
                    SET _field = SUBSTRING_INDEX(_field, "}", 1);
                    RETURN TRIM(
                    BOTH '"' FROM SUBSTRING_INDEX(
                        SUBSTRING_INDEX(
                        SUBSTRING_INDEX(
                            _field,
                            CONCAT(
                            '"',
                            SUBSTRING_INDEX(_variable,'$.', - 1),
                            '":'
                            ),
                            - 1
                        ),
                        ',"',
                        1
                        ),
                        ':',
                        -1
                    )
                    ) ;
                ELSE  
                    SET arrayValue = json_array_value(_field, arrayName);
                    WHILE X < (LENGTH(_variable) - LENGTH(REPLACE(_variable, '.', ""))) DO
                        IF(LENGTH(_variable) - LENGTH(REPLACE(_variable, '.', ""))>X) THEN
                            SET arrayName = SUBSTRING_INDEX(SUBSTRING_INDEX(_variable, '.', X+2),'.',-1);
                        END IF;
                        IF(arrayName<>'') THEN
                            SET arrayValue = json_array_value(arrayValue, arrayName);
                        END IF;
                        SET X = X + 1;
                    END WHILE;
                END IF;
                RETURN arrayValue;
                END$$
DELIMITER ;

Auxiliary function (needed by main function):

CREATE FUNCTION `json_array_value`(
  _field  TEXT,
  arrayName VARCHAR (255)
) RETURNS TEXT CHARSET latin1
BEGIN
                DECLARE arrayValue, arrayValueTillDelimit TEXT;
                DECLARE arrayStartDelimiter, arrayEndDelimiter VARCHAR(10);
                DECLARE arrayCountDelimiter INT;
                DECLARE countBracketLeft, countBracketRight INT DEFAULT 0;
                DECLARE X INT DEFAULT 0;
                DECLARE arrayNameQuoted VARCHAR(255);
                SET arrayNameQuoted = CONCAT('"',arrayName,'"');
                /*check arrayname exist*/
                IF(LOCATE(arrayNameQuoted,_field)= 0) THEN 
                    RETURN NULL;    
                ELSE
                    /*get value behind arrayName1*/
                    SET _field = SUBSTRING(_field,1,LENGTH(_field)-1);
                    SET arrayValue = SUBSTRING(_field, LOCATE(arrayNameQuoted,_field)+LENGTH(arrayNameQuoted)+1, LENGTH(_field));
                    /*get json delimiter*/
                    SET arrayStartDelimiter = LEFT(arrayValue, 1);
                    IF(arrayStartDelimiter='{') THEN
                        SET arrayEndDelimiter = '}';                            
                        loopBrackets: WHILE X < (LENGTH(arrayValue)) DO                 
                            SET countBracketLeft = countBracketLeft +IF(SUBSTRING(arrayValue,X,1)=arrayStartDelimiter,1,0);
                        SET countBracketRight = countBracketRight +IF(SUBSTRING(arrayValue,X,1)=arrayEndDelimiter,1,0);
                        IF(countBracketLeft<>0 AND countBracketLeft=countBracketRight) THEN
                            SET arrayCountDelimiter = X;
                            LEAVE loopBrackets;
                        ELSE
                            SET X = X + 1;
                        END IF;
                        END WHILE;
                                ELSEIF(arrayStartDelimiter='[') THEN
                                    SET arrayEndDelimiter = ']';
                                    SET arrayCountDelimiter = LENGTH(SUBSTRING_INDEX(arrayValue, arrayEndDelimiter, 0));
                                 ELSEIF(arrayStartDelimiter='"') THEN
                                    SET arrayEndDelimiter = '"';
                                    SET arrayCountDelimiter = LENGTH(SUBSTRING_INDEX(arrayValue, arrayEndDelimiter, 0));
                                ELSE 
                                    SET arrayStartDelimiter = "";
                                    IF((LOCATE(",",arrayValue)> LOCATE("}",arrayValue))) THEN
                        SET arrayEndDelimiter = ",";
                        ELSE
                        SET arrayEndDelimiter = "}";
                        END IF;
                        SET arrayCountDelimiter = LENGTH(SUBSTRING_INDEX(arrayValue, arrayEndDelimiter, 0));
                    END IF;
                    SET arrayValueTillDelimit = SUBSTRING(arrayValue, 1, arrayCountDelimiter);
                    SET arrayCountDelimiter = LENGTH(arrayValueTillDelimit) - LENGTH(REPLACE(arrayValueTillDelimit, arrayStartDelimiter, ""));
                    SET arrayValue = SUBSTR(arrayValue,LENGTH(arrayStartDelimiter)+1);
                    IF(arrayStartDelimiter='{') THEN
                        SET arrayValue = SUBSTRING_INDEX(arrayValue, arrayEndDelimiter, arrayCountDelimiter);
                    ELSE
                        SET arrayValue = SUBSTRING_INDEX(arrayValue, arrayEndDelimiter, arrayCountDelimiter+1);
                    END IF;
                    RETURN (arrayValue);
                END IF;
                END$$
DELIMITER ;
标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!