categorical-data

I have been advised that it is best to order categorical variables where appropriate (e.g. short less than medium less than long). I am wondering, what is the specific advantage of treating a categorical variable as ordered as opposed to just simple categorical, in the context of modelling it as an explanatory variable? What does it mean mathematically (in lay terms preferably!)? Many thanks! Among other things, it allows you to compare values from those factors: > ord.fac <- ordered(c("small", "medium", "large"), levels=c("small", "medium", "large")) > fac <- factor(c("small", "medium",

I'm running some models in Python, with data subset on categories. For memory usage, and preprocessing, all the categorical variables are stored as category data type. For each level of a categorical variable in my 'group by' column, I am running a regression, where I need to reset all my categorical variables to those that are present in that subset. I am currently doing this using .cat.remove_unused_categories() , which is taking nearly 50% of my total runtime. At the moment, the worst offender is my grouping column, others are not taking as much time (as I guess there are not as many levels

While working with the DecisionTreeClassifier I visualized it using graphviz , and I have to say, to my astonishment, it seems it takes categorical data and uses it as continuous data. All my features are categorical and for example you can see the following tree (please note that the first feature, X[0], has 6 possible values 0, 1, 2, 3, 4, 5: From what I found here the class uses a tree class which is a binary tree, so it is a limitation in sklearn. Anyone knows a way that I am missing to use the tree categorically? (I know it is not better for the task but as I need categories currently I

Let's say that I have the following code, which comes from here : gender = ['male','male','female','male','female'] import matplotlib.pyplot as plt from collections import Counter c = Counter(gender) men = c['male'] women = c['female'] bar_heights = (men, women) x = (1, 2) fig, ax = plt.subplots() width = 0.4 ax.bar(x, bar_heights, width) ax.set_xlim((0, 3)) ax.set_ylim((0, max(men, women)*1.1)) ax.set_xticks([i+width/2 for i in x]) ax.set_xticklabels(['male', 'female']) plt.show() How could the categories male and female be plotted on the y-axis, as opposed to the x-axis? Perhaps you're

I have a data set called data which has 481 092 rows. I split data into two equal halves: The first halve (row 1: 240 546) is called train and was used for the glm() ; the second halve (row 240 547 : 481 092) is called test and should be used to validate the model; Then I started the regression: testreg <- glm(train$returnShipment ~ train$size + train$color + train$price + train$manufacturerID + train$salutation + train$state + train$age + train$deliverytime, family=binomial(link="logit"), data=train) Now the prediction: prediction <- predict.glm(testreg, newdata=test, type="response") gives