plotly not creating linear trend line

Question

In creating a trend line for a scatter plot, I am using add_trace to add a linear trend line.

When the data only has one "series" of data, i.e. there is only one group of coordinates, the code below works fine. However, when I introduce a number of series, the "trend line" looks like this:

Here is the relevant part of the code:

    p <- plot_ly(filteredFull(), x=Relative.Time.Progress, y=cumul.ans.keystroke,
                 mode='markers', color=KeystrokeRate, size=KeystrokeRate,
                 marker=list(sizeref=100), type='scatter', 
                           hoverinfo='text', text=paste("token: ",Token, "Keystrokes: ",
                                                          KeystrokeCount)) %>%
        layout(
          xaxis=list(range=c(0,1)),
          yaxis=list(range=c(0,max(filteredFull()$cumul.ans.keystroke)))
        )

     lm.all <- lm(cumul.ans.keystroke ~ Relative.Time.Progress,
              data=df)
      observe(print(summary(lm.all)))
      p <- add_trace(p, y=fitted(lm.all), x=Relative.Time.Progress,
                     mode='lines') %>%
        layout(
          xaxis= list(range = c(0,1))
        )
p

I can add more code, or try to make a minimal working example, if necessary. However, I'm hoping that this is a famililar problem that is obvious from the code.

Answer 1

I think you'll need to specify the data = ... argument in add_trace(p, y=fitted(lm.all), x=Relative.Time.Progress, mode='lines').

The first trace seems to be a subset but the second trace uses the regression fitted values which are obtained by fitting a regression model to the entire dataset.

There might be a mismatch between Relative.Time.Progress in filteredFull() vs df.

Here's an example. Hopefully helps...

library(plotly)
df <- diamonds[sample(1:nrow(diamonds), size = 500),]

fit <- lm(price ~ carat, data = df)

df1 <- df %>% filter(cut == "Ideal")

plot_ly(df1, x = carat, y = price, mode = "markers") %>% 
  add_trace(x = carat, y = fitted(fit), mode = "lines")

plot_ly(df1, x = carat, y = price, mode = "markers") %>% 
  add_trace(data = df, x = carat, y = fitted(fit), mode = "lines")

Answer 2

It changed now a bit, the following code should work fine:

df <- diamonds[sample(1:nrow(diamonds), size = 500),]
fit <- lm(price ~ carat, data = df)
df1 <- df %>% filter(cut == "Ideal")
plot_ly() %>%
    add_trace(data = df1, x = ~carat, y = ~price, mode = "markers") %>% 
    add_trace(data = df, x = ~carat, y = fitted(fit), mode = "lines")

Need to start with empty plotly and add traces.