Updated

Chalkboard

Chalkboard is an all-in-one code editor and drawing application designed to help educators explain concepts through both diagramming solutions and executing code in one place. This project was started as my "Solo Project" as part of the Codesmith part-time remote immersive program. The application was developed by me over the course of about two weeks.

For our solo project, we were tasked with developing any application that we wanted as long as it had some sort of CRUD functionality. I went with this idea because I knew it would be the ultimate test of my React knowledge - and that I would face many challenges associated with state management. I also genuinely feel like an application with this functionality should exist - not just for education purposes, but it would also be great for coding interviews held over the internet - since the candidate would be able to "whiteboard" (hence the name chalkboard) through their solution as well as code and test it.

This application is loaded with features and I will probably continue to add some over time. I am going to try and explain these features, including how I coded them up, each one at a time.

Drawing functionality

The very first feature I implemented was the ability to draw SVG lines. Early versions of the application were just basic lines, and then I discovered the NPM package perfect-freehand which gives the lines that natural look and feel through simulated pressure. I also had originally tried to use the built in Canvas API, but quickly realized that it would not be sufficient for all the features I desired to implement. So even though I may use the term "canvas" here, the application contains no HTML canvas element.

The basic idea behind the creation of a line is this:

1. The pointer down event is fired when the user clicks. A PaintableSVG component is added to the application's state and rendered on the screen. This event is handled by the ComponentCanvas component, and then passed to the newly created PaintableSVG component as the prop "createEvent".

  const handlePointerDown = (event: React.PointerEvent<HTMLDivElement>) => {
    if (!activeComponent) {
      return;
    }
    if (event.buttons !== 1) return;

    const randomId = Math.random().toString(36).slice(2, 7);

    addComponent({
      type: activeComponent,
      props: {
        ...activeComponentProps,
        createEvent: event,
      },
      id: `${activeComponent}-${randomId}`,
      data: [],
    });
  };

2. The pointer move event is fired when the user moves the pointer across the screen. This event is read by the PaintableSVG element directly, where it sets its own points based on the updated clientX and clientY properties of the event. These points are continuously fed into perfect-freehand's algorithm to generate an SVG graphic that is displayed.

const handlePointerMove = (event: React.PointerEvent<SVGSVGElement>) => {
  if (!isDrawing) return;
  if (event.buttons !== 1) return;

  setPoints((points) => [
    ...points,
    [
      event.clientX - canvasRect.left,
      event.clientY - canvasRect.top,
      event.pressure,
    ],
  ]);
};
const stroke = getStroke(points, defaultOptions);
const pathData = getSvgPathFromStroke(stroke);

3. The pointer up event is fired when the user releases the mouse or touch. The points are finalized and saved in the application's main state, and the component is no longer in a drawing state.

const handlePointerUp = (event: React.PointerEvent<SVGSVGElement>) => {
  if (!isDrawing) return;
  setIsDrawing(false);
  setData({ points });
};

Using the prop "createEvent", the PaintableSVG component can determine whether or not it was just created and set the state of isDrawing appropriately. This becomes useful later, when the same component can be easily constructed after its data is fetched from the database as a saved canvas.

Other components

The other components that can be drawn on the canvas are the rectangle, the code editor, text, and images (via command or ctrl v). All four of these components are wrapped by the same PaintableDiv component, which handles most of the dirty work - such as the component's position, size, and transform. This component functions in a very similar way to the PaintableSVG - by receiving a createEvent and modifying its data based on the pointer move and pointer up events that follow. In fact, both components receive the same props and therefore are both classified as PaintableComponents through a common type interface.

export interface PaintableComponentProps {
  createEvent: React.PointerEvent<HTMLDivElement> | null;
  color: string;
  id: string;
}

export type PaintableComponent = React.FC<PaintableComponentProps>;

It's actually through these common interfaces on which a lot of the logic used to store the data for the entire canvas is based. To make sure these components can be recreated from the database, each component type is mapped to a string.

export type PaintableComponentMap = Record<string, PaintableComponent>;

const defaultPaintableComponentMap: PaintableComponentMap = {
  svg: PaintableSVG,
  div: PaintableDiv,
  code: PaintableCodeEditor,
  text: PaintableText,
  paste: ClipboardComponent,
  none: () => null,
};

The current active component (the one the user is currently drawing or about to draw with) as well as all the components that currently exist on the canvas, are managed in Zustand stores.

  const { activeComponent, activeComponentProps } = useActiveComponentStore(
    (state) => ({
      activeComponent: state.activeComponent,
      activeComponentProps: state.activeComponentProps,
    })
  );

  const { components, addComponent, updateComponent } = useChalkboardDataStore(
    (state) => ({
      addComponent: state.addComponent,
      components: state.chalkboardComponents,
      updateComponent: state.updateComponent,
    })
  );

The ChalkboardDataStore is a very large Zustand store that is the single source of truth for everything that you see on the canvas - and holds data pertaining to each component's position, transform, size, as well as path points for the PaintableSVG, all the written code in each PaintableCodeEditor, and the image url or text for the ClipboardComponent. It holds metadata for the canvas such as the title, the document id if it has been saved in the database, and the logic necessary to persist this data on the backend. It is a very large store.

export interface ChalkboardDataStore {
  chalkboardComponents: PaintableComponentData[];
  chalkboardId: string | null;
  chalkboardTitle: string;
  resetChalkboard: () => void;
  loadFromDatabase: (
    fetcher: (input: RequestInfo, init?: RequestInit) => Promise<Response>,
    chalkboardId: string,
    onSuccess?: (message: string) => void,
    onError?: (error: string) => void
  ) => void;
  loadFromLocalStorage: () => void;
  saveToDatabase: (
    fetcher: (input: RequestInfo, init?: RequestInit) => Promise<Response>,
    saveAsNew: boolean,
    onSuccess?: (message: string) => void,
    onError?: (error: string) => void
  ) => void;
  saveToLocalStorage: () => void;
  addComponent: (component: PaintableComponentData) => void;
  removeComponent: (componentId: string) => void;
  updateComponent: (
    componentId: string,
    update: Partial<PaintableComponentData>
  ) => void;
  updateComponentData: (componentId: string, update: any) => void;
  moveComponent: (componentId: string, newIndex?: number) => void;
  updateTitle: (title: string) => void;
  getComponent: (componentId: string) => PaintableComponentData | undefined;
}

The use of Zustand in this application has made saving and loading all of this data much easier to work with. One challenge with this data though is the dynamic nature of the nested component data.

export interface PaintableComponentData {
  type: string;
  id: string;
  props?: PaintableComponentProps & any;
  data: any;
}

The PaintableComponentData represents all the information needed to persist a component on the canvas. As you can see here though I've broken a general rule that I don't like to break - I've used the any type. One day I may go back and refactor this to use generics somehow, but for now I'm using any because of how the data is persisted in MongoDB. Here is the schema used to store all the data for a chalkboard:

const canvasSchema = new mongoose.Schema<CanvasData>({
  title: {
    type: String,
    required: true,
  },
  userEmail: {
    type: String,
    required: true,
  },
  components: [
    {
      type: {
        type: String,
        required: true,
      },
      props: mongoose.Schema.Types.Mixed,
      data: mongoose.Schema.Types.Mixed,
      id: {
        type: String,
        required: true,
      },
    },
  ],
  updatedAt: {
    type: Date,
    default: Date.now,
  },
});

As you can see here, props and data are Mixed. This is because something like a PaintableCodeEditor and a PaintableSVG are going to have drastically different shapes for these properties. However, this does mean that the entire components array needs overwritten on every save, as demonstrated below with the canvas.set() function.

if (method === 'PATCH') {
    const saveData = { ...req.body, userEmail: session.user.email };
    console.log('Attempting to update canvas with id: ' + canvasId);

    try {
      const canvas = await CanvasModel.findById(canvasId);
      if (canvas.userEmail !== session.user.email) {
        console.log(
          'Canvas found but unable to verify user. Canvas user: ' +
            canvas.userEmail +
            ', current user: ' +
            session.user.email
        );
        return res.status(401).json({ message: 'Unauthorized' });
      }

      canvas.title = saveData.title;
      canvas.set({ components: saveData.components });
      canvas.updatedAt = new Date();

      await canvas.save({});

      return res.status(200).json({ success: true, data: canvas });
    } catch (error) {
      console.log('Error updating canvas: ', error);
      return res.status(400).json({ success: false });
    }
  }

I won't say how long it took me to figure out that Mixed data types in Mongoose don't update when calling .save().

Selecting and transforming

After getting some suggestions from ChatGPT (which I've found to be great for finding jumping-off points, but not so great for implementation details) I went with using the react-selecto and react-moveable packages for implementing selection and transform manipulation of each component. These are React implementations of vanilla JS libraries, so navigating the documentation for each of them was a little challenging. However, I implemented them through the use of Zustand stores and higher order components (HOCs).

For selection, finding the right configuration options for the component was probably the biggest challenge. My SelectionManager component essentially just exports the Selecto component with a whole bunch of configuration options that change dynamically.

return (
    <Selecto
      dragCondition={(e) => canvasRef?.current && !activeComponent && !hasFocus}
      container={canvasRef?.current}
      selectableTargets={selectableElements}
      toggleContinueSelect={['shift']}
      // This enables the user to move a group of elements when shift is not held. Without it, the group would be unselected.
      selectByClick={selectedElements.length <= 1 || shiftKeyHeld}
      selectFromInside={false}
      onSelect={(e) => {
        e.added.forEach((el) => {
          addSelectedElement(el);
        });
        e.removed.forEach((el) => {
          removeSelectedElement(el);
        });
      }}
    />
  );

The Zustand store for selection holds both a list of selectable components (those that can be selected but are not yet) and a list of selected components. It's also useful for turning selection on and off altogether.

interface SelectionStore {
  selectableElements: HTMLElement[];
  addSelectableElement: (element: HTMLElement) => void;
  removeSelectableElement: (id: string) => void;
  selectedElements: (HTMLElement | SVGElement)[];
  addSelectedElement: (element: HTMLElement | SVGElement) => void;
  removeSelectedElement: (element: HTMLElement | SVGElement) => void;
  removeSelectedElementById: (id: string) => void;
  selectionEnabled: boolean;
  setSelectionEnabled: (enabled: boolean) => void;
  clearSelection: () => void;
}

And lastly, the HOC for selection simply adds each component to the list of selectable components when it is created. I feel like this is similar to back in my C#/Unity days when I used to maintain static lists in classes to which each instantiated object of that class could add or remove itself.

const withSelectable = <P extends PaintableComponentProps>(
  WrappedComponent: PaintableComponent
) => {
  return (props: P) => {
    const { addSelectableElement, removeSelectableElement } = useSelectionStore(
      (state) => ({
        addSelectableElement: state.addSelectableElement,
        removeSelectableElement: state.removeSelectableElement,
      })
    );

    const wrappedComponentRef = React.createRef<HTMLElement>();
    React.useEffect(() => {
      const componentId = wrappedComponentRef.current?.id;
      addSelectableElement(wrappedComponentRef.current);
      return () => {
        removeSelectableElement(componentId);
      };
    }, []);

    return <WrappedComponent ref={wrappedComponentRef} {...props} />;
  };
};

export default withSelectable;

For manipulating the transform of a component, I also create a MoveableManager component which contained the Moveable component with all its configuration options - and the list is much longer. Essentially it applies the transform options to anything in the selectedElements of the SelectionStore.

return (
    <Moveable
      targets={selectedElements}
      origin={false}
      draggable={true}
      onDrag={({ target, transform }) => {
        target.style.transform = transform;
        updateComponentData(target.id, { transform });
      }}
      onDragGroup={({ events }) => {
        events.forEach(({ target, transform }) => {
          target.style.transform = transform;
          updateComponentData(target.id, { transform });
        });
      }}
      // SVG path elements should be scaled versus resized
      scalable={
        selectedElements.length === 1 &&
        selectedElements[0].id.startsWith('svg')
      }
      onScale={({ target, transform }) => {
        if (!target.id.startsWith('svg')) {
          return;
        }
        updateComponentData(target.id, { transform });
      }}
      resizable={
        selectedElements.length === 1 &&
        !selectedElements[0].id.startsWith('svg')
      }
      onResize={({ target, width, height }) => {
        if (target.id.startsWith('svg')) {
          return;
        }
        // This immediately sets width and height, and also updates the state. The redundancy is necessary because the state update is slower and causes a noticeable lag, but is still needed to save the data.
        target.style.width = `${width}px`;
        target.style.height = `${height}px`;
        updateComponentData(target.id, { width, height });
      }}
      rotatable={selectedElements.length === 1}
      onRotate={({ target, transform }) => {
        updateComponentData(target.id, { transform });
      }}
      keepRatio={shiftKeyHeld}
    />
  );

So far, however, I have only been able to get dragging to work with multiple selected elements. Scaling does not work. I'm also not a huge fan of using scale for SVG but resize for everything else - but the alternative does not produce desired behavior. Scaling a code editor, for example, also scales the text and everything else. Resizing an SVG just updates the bounding box but not the path. I definitely think I could do better work on this one, but it's functional for now.