数据库重构_有效重构繁重的数据库接口

最新推荐文章于 2022-05-06 16:38:45 发布

郝ren

最新推荐文章于 2022-05-06 16:38:45 发布

阅读量157

点赞数

文章标签：数据库 java mysql sql linux

原文链接：https://blog.maddevs.io/effective-refactoring-of-heavy-database-interface-2c355498afe7

版权

数据库重构

This story is about pain, agony, and denial of ready-made solutions. It is also about changes that improve the code’s readability and help the development team stay happy. The object of this post is an interface that helps a program communicate with a database.

这个故事讲述痛苦，痛苦和拒绝现成的解决方案。它还涉及提高代码的可读性并帮助开发团队保持满意的更改。这篇文章的目的是一个接口，可以帮助程序与数据库进行通信。

Disclaimer: If we would use various ORMs such as Gorm in this project, most probably we would not face this issue, yet, we decided to write our implementation, so this created the issue and therefore this post.

免责声明：如果我们将使用各种奥姆斯如格姆在这个项目中，最有可能我们就不会面临这样的问题，但是，我们决定写我们的实现，所以这个创建的问题，因此这个职位。

// IStorage represents database methods
type IStorage interface {
	IfUserCanBeDeleted(ctx context.Context, userID int64) (bool, error)
	CreateUser(ctx context.Context, u *model.User) (model.User, error)
	CreateProfile(ctx context.Context, item *model.Profile) (*model.Profile, error)
	DeleteUser(ctx context.Context, id int64) error
	MarkUserAsDeleted(ctx context.Context, id int64, reason string) error
	ListProfiles(ctx context.Context) ([]model.Profile, error)
	UpdateProfile(ctx context.Context, item *model.Profile) error
	GetUserByID(ctx context.Context, id int64) (model.User, error)
	GetPinCodeByUserID(ctx context.Context, id int64) (string, error)
	GetShortUserByID(ctx context.Context, id int64) (model.ShortUser, error)
	UpdateProfilePersonID(ctx context.Context, accID, personID string) error
	GetUserProfileByID(ctx context.Context, id int64) (*model.Profile, error)
	GetUserProfileByStripeAccountID(ctx context.Context, stripeAccountID string) (*model.Profile, error)
	GetUserProfileByStripeCustomerID(ctx context.Context, stripeCustomerID string) (*model.Profile, error)
	GetUserIDByStripeAccount(ctx context.Context, stripeAccount string) (int64, error)
	GetAgreementIDByPaymentIntentID(ctx context.Context, paymentIntentID string) (int64, error)
	GetUserByPhone(ctx context.Context, phone string) (model.User, error)
	CheckUserExists(ctx context.Context, phone string) (bool, error)
	CheckUserExistsDeprecated(ctx context.Context, phones []string) (bool, error)
	CreateAgreement(ctx context.Context, a *model.Agreement) (model.Agreement, error)
	GetAgreementByPaymentIntentID(ctx context.Context, paymentIntentID string) (model.Agreement, error)
	GetAgreementByResolutionServicesPaymentIntentID(ctx context.Context, paymentIntentClientSecret string) (model.Agreement, error)
	UpdateAgreementResolutionServicePI(ctx context.Context, id int64, piClientSecret string) error
	DeleteAgreement(ctx context.Context, id int64) error
	GetPaidAgreementsForBuyer(ctx context.Context, buyerID int64) ([]model.Agreement, error)
	UpdateAgreementStatus(ctx context.Context, a *model.Agreement) error
	GetAgreementByID(ctx context.Context, id int64) (model.Agreement, error)
	GetUserAgreements(ctx context.Context, filter *model.AgreementFilter) ([]model.Agreement, error)
	GetAgreementMilestones(ctx context.Context, agreementID int64) ([]model.Milestone, error)
	ConfirmAgreement(ctx context.Context, agreement model.Agreement) error
	RejectChangesOnAgreement(ctx context.Context, a *model.Agreement) error
	AcceptChangesOnAgreement(ctx context.Context, initialAgreement *model.Agreement, newAgreement model.Agreement, updatedMstList, deletedMstList, newAddedMstList []model.Milestone) error
	UpdateMilestone(ctx context.Context, m model.Milestone) (model.Milestone, error)
	UpdateAgreement(ctx context.Context, a *model.Agreement, updatedMstList, deletedMstList, newAddedMstList []model.Milestone) error
	ContactsList(ctx context.Context, id int64) (model.ContactListForUser, error)
	ContactsCreate(ctx context.Context, contact model.ContactPairShort) error
	ContactsDelete(ctx context.Context, userID int64) error
	ContactsListFriends(ctx context.Context, id int64) (model.ContactListForUser, error)
	GetContactShort(ctx context.Context, id1, id2 int64) (model.ContactPairShort, error)
	GetContactShortByChatID(ctx context.Context, chatID string) (model.ContactPairShort, error)
	GetContactFullByChatID(ctx context.Context, chatID string) (model.ContactPairFull, error)
	GetContactPartialInfoSecond(ctx context.Context, id1, id2 int64) (model.ContactPartialInfoSecond, error)
	GetContactFull(ctx context.Context, id1, id2 int64) (model.ContactPairFull, error)
	ContactsCreateAddRequest(ctx context.Context, idFrom, idTo int64) (model.ContactPairShort, error)
	ContactsUpdateLastMessageTime(ctx context.Context, contact *model.ContactPairShort) error
	ContactExists(ctx context.Context, id1, id2 int64) (bool, error)
	GetUsersReferralCode(ctx context.Context, userID int64) (string, error)
	ContactsUpdate(ctx context.Context, contact model.ContactPairShort) error
	GetMilestoneByID(ctx context.Context, id int64) (model.Milestone, error)
	UpdateMilestoneStatus(ctx context.Context, m *model.Milestone) error
	GetShortAgreement(ctx context.Context, id int64) (model.Agreement, error)
	UpdateUser(ctx context.Context, u *model.User) error
	UpdateUserPinCode(ctx context.Context, u *model.User) error
	SaveConfirmationCode(ctx context.Context, c model.ConfirmationCode) error
	GetConfirmationCodeInfo(ctx context.Context, phone string) (model.ConfirmationCode, error)
	AgreementMilestonesClosed(ctx context.Context, agreementID int64) (bool, error)
	CreateUserDeviceToken(ctx context.Context, dt *model.UserDeviceToken) error
	RemoveUserToken(ctx context.Context, userID int64) error
	GetNotification(ctx context.Context, id int64) (model.Notification, error)
	ListNotification(ctx context.Context, userID int64) ([]model.Notification, error)
	CreateNotification(ctx context.Context, n *model.Notification) (model.Notification, error)
	GetUserFCMToken(ctx context.Context, userID int64) (string, error)
	SubmitAgreementForInternalResolution(ctx context.Context, id int64, piClientSecret string) error
	SetAgreementResolution(ctx context.Context, id int64, resolution string) error
	DenyAgreement(ctx context.Context, a model.Agreement) error
	GetAgreementOfMilestone(ctx context.Context, milestoneID int64) (model.Agreement, error)
	SoftDeleteMilestone(ctx context.Context, agreementID, milestoneID int64) error
	SetMilestoneToDelete(ctx context.Context, agreementID, milestoneID int64) error
	SetMilestoneToUpdate(ctx context.Context, milestoneID int64) error
	ConfirmPhone(ctx context.Context, phone string) error
	IsPhoneVerified(ctx context.Context, phone string) (bool, error)
	RemoveConfirmationCode(ctx context.Context, phone string) error
	GetAllShortUsers(ctx context.Context) ([]model.ShortUser, error)
	SetNotificationAsRead(ctx context.Context, id int64) error
	GetUnreadNotificationsCount(ctx context.Context, receiverID int64) (int, error)
	ResetUnreadNotifications(ctx context.Context, receiverID int64) error
	SaveWebhookEvent(ctx context.Context, item *model.WebhookEvent) (model.WebhookEvent, error)
	UpdateWebhookEvent(ctx context.Context, item *model.WebhookEvent) error
	FindWebhookEventBy(ctx context.Context, source, externalID string) (model.WebhookEvent, error)
	GetFullAgreementByID(ctx context.Context, id, tokenUserID int64) (model.Agreement, error)
	ReferralGetBonusByUserID(ctx context.Context, userID int64) (int64, error)
	ReferralAddBonusToUsersBonuses(ctx context.Context, userID, bonus int64) error
	ReferralExists(ctx context.Context, refCode string) (bool, error)
	GetReferral(ctx context.Context, code string) (model.Referral, error)
	AddReferralProgram(ctx context.Context, rp model.ReferralProgram) error
	DeleteReferralPrograms(ctx context.Context) error
	GetReferralProgram(ctx context.Context, code string) (*model.ReferralProgram, error)
	ListReferralPrograms(ctx context.Context) ([]model.ReferralProgram, error)
	ListAgreementsApplicableForReferral(ctx context.Context, code string) ([]model.Agreement, error)
	ReferralUserAlreadyUsedRefferalCode(ctx context.Context, userID int64) (bool, error)
	IfMilestoneWasUsedInReferral(ctx context.Context, milestoneID int64, code string) (bool, error)
	MarkMilestoneAsUsedInReferral(ctx context.Context, milestoneID int64, code string) error
	GetReferralByUserID(ctx context.Context, userID int64) (model.Referral, error)
	CreateMilestoneChanges(ctx context.Context, milestone *model.Milestone) error
	GetMilestoneChanges(ctx context.Context, milestoneID int64) (*model.Milestone, error)
	DeleteMilestoneChanges(ctx context.Context, milestoneID int64) error
	ReferralGetUserIDByCode(ctx context.Context, code string) (int64, error)
	CreatePayout(ctx context.Context, item model.Payout) (model.Payout, error)
	DeletePayout(ctx context.Context, id int64) error
	ListPayouts(ctx context.Context, userID int64) ([]model.Payout, error)
	ListPayoutsWithStatus(ctx context.Context, status model.PayoutStatus) ([]model.Payout, error)
	UpdatePayoutStripeID(ctx context.Context, item model.Payout) error
	UpdatePayoutStatus(ctx context.Context, item model.Payout) (model.Payout, error)
	GetPayoutByMilestoneID(ctx context.Context, milestoneID int64) (model.Payout, error)
	GetPayoutByStripeID(ctx context.Context, payout string) (model.Payout, error)
	CreateFeedback(ctx context.Context, f *feedback.Feedback) (*feedback.Feedback, error)
	GetUserIDByEmail(ctx context.Context, email string) (int64, error)
	CreatePlatformTransfer(ctx context.Context, item model.PlatformTransfer) (model.PlatformTransfer, error)
	ListAgreementPlatformTransfers(ctx context.Context, agreementID int64) ([]model.PlatformTransfer, error)
	DisputeAgreement(ctx context.Context, agreementID int64, milestoneIDs []int64) error
	CreateBlockedUser(ctx context.Context, item model.BlockedUser) (model.BlockedUser, error)
	CheckIfUserIsBlocked(ctx context.Context, userID int64) (bool, error)
	UnblockUser(ctx context.Context, stripeAccountID string) error
	CreateUserRefreshToken(ctx context.Context, rt model.UserRefreshToken) error
	GetUserRefreshToken(ctx context.Context, userID int64) (string, error)
	UpdateUserRefreshToken(ctx context.Context, rt model.UserRefreshToken) error
	UpdateUserDeviceToken(ctx context.Context, dt model.UserDeviceToken) error
	GetConfirmedMilestones(ctx context.Context) ([]model.Milestone, error)
	CollaboratorsGetAllPhonesWithContactStatus(ctx context.Context, callerID int64) ([]model.PhoneWithContactStatus, error)
	SetShowRejectedChangesFalse(ctx context.Context, changesID int64) error
	CreateAgreementChanges(ctx context.Context, item model.ChangesOnAgreement) (model.ChangesOnAgreement, error)
	GetLatestAgreementChanges(ctx context.Context, agreementID int64) (model.ChangesOnAgreement, error)
	DeleteAgreementChanges(ctx context.Context, id int64) error
	DeleteAgreementChangesByAgreementID(ctx context.Context, id int64) error
	CancelAgreementChanges(ctx context.Context, agreement model.Agreement, changesID int64) error
	GetAgreementChangesByID(ctx context.Context, id int64) (model.ChangesOnAgreement, error)
	SetChangesViewed(ctx context.Context, changesOnAgrID int64) error
	CheckIfAgreementHasPendingChanges(ctx context.Context, agreementID int64) (bool, error)
	GetDataOfMilestoneToInspect(ctx context.Context, previousRunTime time.Time) ([]model.MilestoneToInpect, error)
	CreateMilestone(ctx context.Context, m *model.Milestone) (model.Milestone, error)
	SetLatestLoggedUserDevice(ctx context.Context, u model.LatestLoggedUserDevice) (model.LatestLoggedUserDevice, error)
	GetLatestLoggedUserDevice(ctx context.Context, userID int64) (model.LatestLoggedUserDevice, error)
}

The problem with this interface was its size — 130+ methods in one single interface! That’s a lot of methods and that is not what SOLID interface should look like. And since we develop in Go, we have to know (and follow) one of the Go proverbs which are:

该接口的问题在于它的大小-一个接口中有130多种方法！这有很多方法，而不是SOLID接口应具有的外观。并且由于我们在Go中发展，我们必须知道(并遵循)以下Go谚语之一：

The bigger the interface, the weaker the abstraction. © Rob Pike

接口越大，抽象性越弱。 ©罗伯·派克

The further we developed the project, the heavier this interface grew and soon it became clear that to continue the development with fewer bugs, less time spent understanding the code, and more comfort, this interface should be refactored. We as a team could not use this interface with flexibility. We could not tell from the first glance what it does since it does everything. And that forced me to start its refactoring. Which is what I want to share with you.

我们对项目的开发越深入，该接口的负担就越大，并且很快就可以清楚地知道，要重构该接口，就应该以更少的错误，更少的时间来理解代码以及更加舒适地继续开发。我们团队无法灵活使用此界面。乍看之下，我们无法判断它在做什么，因为它可以完成所有工作。这迫使我开始进行重构。我想与您分享

步骤0：在任何重构之前用测试覆盖代码(API) (Step 0: Cover Code (API) with Tests before any Refactoring)

This is crucial, since dealing with interfaces, abstractions, and refactoring without tests that cover API logic makes no good. I would say that it can do exactly opposite — bring a lot of problems to your code since every change you make to the interface will result in 20+ files being changed. And if you do not have solid tests, there is a high chance you break something or create bugs. Please, be cautious!

这很关键，因为在没有涉及API逻辑的测试的情况下处理接口，抽象和重构都没有用。我要说的是，它的作用恰恰相反—给您的代码带来很多问题，因为对接口进行的每次更改都会导致20多个文件被更改。而且，如果您没有可靠的测试，则很有可能会破坏某些东西或创建错误。请小心！

第1步：想象结果如何 (Step 1: Imagine What the Result Would Look Like)

I decided to concentrate on common aggregates that my project deals with. After some time of thinking and looking through the entire list of functions, I outlined how the future interface would look like and this is what I came up with:

我决定专注于我的项目处理的常见总量。经过一段时间的思考并仔细阅读了整个功能列表，我概述了将来的界面的外观，这就是我的想法：

// IStorage represents database methods
type IStorage interface {
	User() User
	Profile() Profile
	Agreement() Agreement
	AgreementChanges() AgreementChanges
	Milestone() Milestone
	Contact() Contact
	Notification() Notification
	Payout() Payout
	WebhookEvent() WebhookEvent
	Referral() Referral
	VerificationCode() VerificationCode
	Feedback() Feedback
	PlatformTransfer() PlatformTransfer
}

This would allow writing the following code instead of just reference one of 130+ methods from the interface:

这将允许编写以下代码，而不是仅从接口引用130多种方法之一：

user, err := api.Storage.GetUserByID(ctx, userID)
err := api.Storage.DenyAgreement(ctx, agreement)
err := api.Storage.UpdateUserDeviceToken(ctx, model.UserDeviceToken{...})

After refactoring:

重构后：

user, err := api.Storage.User().Get(ctx, userID)
err := api.Storage.Agreement().Deny(ctx, agreement)
err := api.Storage.User().UpdateDeviceToken(ctx, model.UserDeviceToken{...})

As you can see, this interface consists of multiple smaller interfaces each based on certain aggregates (users, agreements, etc.) and doing something with that aggregate. Reading such constructions is a much better experience and at the same time, it is more convenient since if you ever need to do anything with a user, you know where to search for the right methods and consider if they even exist.

如您所见，此接口由多个较小的接口组成，每个接口均基于某些聚合(用户，协议等)，并对该聚合进行操作。阅读这样的结构是更好的体验，同时，它也更方便，因为如果您需要与用户做任何事情，您就知道在哪里寻找正确的方法并考虑它们是否存在。

步骤2：逐步执行变更 (Step 2: Implement Changes Step by Step)

Having interface with 130+ methods makes it very complicated to do refactor in “once and for all” style. There are so many changes that turn every merge request into 50+ files changes. So, the next step, therefore, should be breaking the interface step by step, one aggregate after another and committing those changes often to make small, understandable merge request and making sure everything still works as expected (remember step 0!) For this I first break down methods into small portions

具有130多种方法的接口使得以“一劳永逸”的方式进行重构非常复杂。有如此多的更改，使每个合并请求变成50多个文件更改。因此，因此，下一步应该是逐步打破接口，一个接一个地聚合，并经常提交那些更改以发出小的，可理解的合并请求，并确保一切仍按预期进行(记住步骤0！)。首先将方法分解成小部分

// IStorage represents database methods
type IStorage interface {
	// User() User
	// Profile() Profile
	// Agreement() Agreement
	// AgreementChanges() AgreementChange
	// Milestone() Milestone
	// Contact() Contact
	// Notifications() Notification
	// Payout() Payout
	// WebhookEvents() WebhookEvent
	// Referrals() Referral
	// VerificationCodes() VerificationCode
	// Feedback() Feedback
	// PlatformTransfe() PlatformTransfer


	// this is just a temporary change not to break all the functionality. Will reimplment this
	// for the version mentioned above as a next step with just a small chucks under improvements
	User
	Profile
	Agreement
	AgreementChange
	Milestone
	Contact
	Notification
	Payout
	WebhookEvent
	Referral
	VerificationCode
	Feedback
	PlatformTransfer
}

So, I created several sub-interfaces and stated that my IStorage interface implements them all. That did not change the code much but laid an important preparatory brick to what I wanted to do next, which is essentially replace my sub-interfaces with separate interfaces with their separate methods in CRUD fashion, adding missing methods & uniting those which are of the same nature. I added new structs as the implementation of those interfaces and replaced old methods with a new one with VSCode search & replace all features.

因此，我创建了几个子接口，并说我的IStorage接口实现了所有子接口。那并没有改变太多代码，但是为我接下来要做的事情准备了一个重要的准备工作，这实际上是用子接口以CRUD方式用单独的方法替换子接口，从而增加了缺失的方法并统一了那些子接口。相同的性质。我添加了新的结构作为这些接口的实现，并用VSCode搜索和替换所有功能的新方法替换了旧方法。

// IStorage represents database methods
type IStorage interface {
	Feedback() Feedback


	// this is just a temporary change not to break all the functionality. Will reimplment this
	// for the version mentioned above as a next step with just a small chucks under improvements
	User
	Profile
	Agreement
	AgreementChange
	Milestone
	Contact
	Notification
	Payout
	WebhookEvent
	Referral
	VerificationCode
	PlatformTransfer
}


func (s *Storage) Feedback() Feedback {
	return &FeedbackClient{cfg: s.cfg, logger: s.logger, pool: s.pool}
}


// Feedback inferface contains methods for feedback manipulation
type Feedback interface {
	Create(ctx context.Context, f *feedback.Feedback) (*feedback.Feedback, error)
}


type FeedbackClient struct {
	cfg    config.Config
	logger *logrus.Entry
	pool   *pgxpool.Pool
}

As a result, I have changed all IStorage sub-interfaces to its interfaces and made all functions more intuitive & understandable. Yes, it took me a while to do that, yet, the result is worth it.

结果，我将所有IStorage子接口更改为其接口，并使所有功能更加直观和易于理解。是的，我花了一段时间才这样做，但是结果值得。

步骤3：清理 (Step 3: Clean up)

Bulk edits with find -> replace all helps save time, but it also creates some side effects where you can rename not relevant function or the logs messages might get a bit silly and hard to understand. This what happened to me after refactoring.

使用find->全部替换进行批量编辑有助于节省时间，但同时也会带来一些副作用，您可以在其中重命名不相关的功能，否则日志消息可能会变得有些愚蠢且难以理解。这是重构后发生在我身上的事情。

err = api.Storage.Profile().Update(ctx, user)
if err != nil {
  api.Logger.WithFields(logrus.Fields{"err": err}).Error("Update failed")
}

When this gets logged we get the message “Update failed” which might not be so clear to us at first glance. Yes, there is a “handler” param that indicates where exactly it happened, but this might not be enough to determine the exact spot. So, my advice here would be to go ahead and review the project’s log messages and other components that might get affected by the refactoring.

记录下来后，我们会收到“更新失败”消息，乍一看我们可能不太清楚。是的，有一个“处理程序”参数可以指示确切的位置，但这可能不足以确定确切的位置。因此，我的建议是继续并检查项目的日志消息和其他可能受到重构影响的组件。

重构的副作用 (Side Effects from Refactoring)

During this process, it became clear that we use a lot of different methods for the same thing (like updating items in various fields in various methods). This creates code smell and in the future, we would avoid that anti-pattern.

在此过程中，很明显，我们对同一事物使用了许多不同的方法(例如，以各种方法更新各个字段中的项目)。这会产生代码异味，将来，我们将避免使用这种反模式。

Also, I have found some functions that were not “fit” in certain places. For example when agreements were highly dependent on referrals in the update. That made no sense after I begin refactoring. Having this “God” interface allowed code to use it, but refactoring showed how terrible it was and made living with this code impossible. I have to re-write some functions to create more natural, intuitive functionality.

另外，我发现某些功能在某些地方不是“合适的”。例如，当协议高度依赖于更新中的引荐。在我开始重构之后，这毫无意义。有了这个“上帝”界面，代码就可以使用它，但是重构显示了它的可怕程度，使得无法使用该代码。我必须重新编写一些功能以创建更自然，直观的功能。

Yes, we have to write some more code, yet, this approach gives us way more to code’s readability, maintainability, and simple design. I would do it after all.

是的，我们必须编写更多代码，但是，这种方法为我们提供了更多方式来提高代码的可读性，可维护性和简单设计。毕竟我会做。

结论 (Conclusion)

If you have a project with a very heavy database interface and you see that it continues to grow, consider refactoring as soon as possible since it would just get worse.

如果您的项目的数据库接口非常繁重，并且您看到它还在继续增长，请考虑尽快进行重构，因为这样会使情况变得更糟。

When refactoring, always check tests first, then decide what the end solution should look like and do partial updates, one component at a time. This will keep your project alive, will decrease chances of introducing new bugs, will keep code reviewers from anger, and keep you happy about the progress.

重构时，请始终先检查测试，然后确定最终解决方案的外观并进行部分更新，一次更新一个组件。这将使您的项目保持活力，减少引入新错误的机会，使代码审阅者免受愤怒，并使您对进度感到满意。

After refactoring large interfaces always check for ‘side effects”. basicallyIf you ever face the same issue, please share your solutions in the comments, since I am really curious about what else I might have done. Also, please share your thoughts on what I could (and still can) do to make it even better. Any feedback is more than welcomed!

重构大型接口后，请务必检查“副作用”。基本上，如果您遇到相同的问题，请在评论中分享您的解决方案，因为我真的很想知道我可能还会做些什么。另外，请分享您对我可以(而且仍然可以)做得更好的想法。任何反馈都值得欢迎！